Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Docs

# Builds the Read the Docs site on every PR that can change it, with warnings as
# errors, so a broken link or a bad toctree entry fails here instead of on RTD
# after merge. Same Python and requirements as .readthedocs.yml. (#137)
#
# Publishing is not done here: RTD builds each channel (stable, vX.Y.Z-alpha,
# vX.Y.Z-beta) when release.yml publishes it - see the docs-publish job there.

# No paths filter: doc/ links into the rest of the repo (tests/, CarMaker/,
# CLAUDE.md, ...), so moving or deleting a file anywhere can dangle a docs link.
# An allow-list would silently miss the next directory someone links to; the
# build is ~1 min, so it just runs.
on:
pull_request:
branches: [dev, main, 'dev_v*', 'beta_v*']

jobs:
sphinx:
runs-on: ubuntu-latest
steps:
# Depth 1 but the whole tree: the link rewriter checks that each target
# exists. ~230 MB; history (the ~1 GB part) is not needed.
- uses: actions/checkout@v4
with:
fetch-depth: 1
submodules: false

- uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Install doc requirements
run: python -m pip install -r doc/requirements_doc.txt

- name: Build (warnings are errors)
run: python -m sphinx -T -W --keep-going -b html doc _build/html
36 changes: 36 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -552,3 +552,39 @@ jobs:
# --clobber replaces only these asset names; the Windows bundle on the
# same rolling release is untouched.
gh release upload "$CHANNEL_TAG" "$want" "$want.sha256" --repo "$GITHUB_REPOSITORY" --clobber

# Read the Docs serves one docs version per channel under the channel's own
# name (stable, vX.Y.Z-beta, vX.Y.Z-alpha), built from the tag published above
# (#137). Ask RTD to rebuild it here, so the docs move exactly when the channel
# does, whatever RTD's webhook makes of a tag that was deleted and recreated.
docs-publish:
name: Rebuild the channel's docs on Read the Docs
needs: build-publish
if: needs.build-publish.outputs.publish == 'true' && (github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && inputs.publish))
runs-on: ubuntu-latest
steps:
- name: Trigger the RTD build for the channel's version
env:
RTD_TOKEN: ${{ secrets.RTD_TOKEN }}
CHANNEL_TAG: ${{ needs.build-publish.outputs.tag }}
run: |
if [ -z "$RTD_TOKEN" ]; then
echo "::warning::No RTD_TOKEN secret - the $CHANNEL_TAG docs were not rebuilt."
exit 0
fi
api=https://app.readthedocs.org/api/v3/projects/real-sim
# RTD learns about the recreated tag from the webhook. Until it has
# synced, the version can be missing or still inactive, so sync + retry.
for attempt in 1 2 3 4 5 6; do
code=$(curl -sS -o resp.json -w '%{http_code}' -X POST \
-H "Authorization: Token $RTD_TOKEN" "$api/versions/$CHANNEL_TAG/builds/")
if [ "$code" = 202 ]; then
echo "RTD build for $CHANNEL_TAG: $(jq -r '.build.urls.build' resp.json)"
exit 0
fi
echo "attempt $attempt: HTTP $code $(cat resp.json)"
curl -sS -o /dev/null -X POST -H "Authorization: Token $RTD_TOKEN" "$api/sync-versions/" || true
sleep 20
done
echo "::error::Read the Docs would not build '$CHANNEL_TAG'. Check the version exists and is active (RTD automation rule for channel tags)."
exit 1
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ build:

sphinx:
configuration: doc/conf.py
fail_on_warning: false
fail_on_warning: true

python:
install:
Expand Down
15 changes: 8 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,14 +91,14 @@ scripts\dispatch\2_core_components.bat
# VISSIM driver models only
scripts\dispatch\3_vissim_components.bat

# CarMaker (auto-generates BuildConfig files based on dependencies.yaml)
powershell -ExecutionPolicy Bypass -File scripts\dispatch\4a_carmaker_components.ps1
# CarMaker executables
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5a_carmaker_components.ps1

# dSPACE libraries
powershell -ExecutionPolicy Bypass -File scripts\dispatch\4b_carmaker_dspace.ps1
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5b_carmaker_dspace.ps1

# MATLAB MEX file
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5_mex_realsim_socket.ps1
powershell -ExecutionPolicy Bypass -File scripts\dispatch\6_mex_realsim_socket.ps1
```

### Build System Features
Expand Down Expand Up @@ -490,7 +490,7 @@ Forward-looking decisions for unifying the virtual-environment bridges (CarMaker
- **`TrafficHelper.cpp` (~1743 lines)** — SUMO and VISSIM logic is fully interleaved; tracked in milestone 0.8.0. Avoid adding to it; route new functionality to helper sub-files instead.
- **`mainTrafficLayer.cpp` (~1396 lines)** — The main() function does too much directly. Issue #113 tracks a refactor. Prefer adding logic in helper classes, not directly in main.
- **`ConfigHelper.cpp` (~1169 lines)** — Has dead `// TODO: add code` branches and commented-out error blocks; safe to clean up but not urgent.
- **`MsgHelper.py` (~28k lines)** — Almost certainly has dead code paths. Any changes there should be narrow and tested; full audit deferred.
- **`MsgHelper.py` (~680 lines)** — Likely has dead code paths. Any changes there should be narrow and tested; full audit deferred.

### No automated CI
- Python unit tests run locally only (`tests/Python/unit/`). Issue #57 tracks GitHub Actions setup. When suggesting test commands, always direct to `python -m pytest tests/Python/unit/ -v` — this is the one simulator-free path.
Expand All @@ -503,8 +503,9 @@ Forward-looking decisions for unifying the virtual-environment bridges (CarMaker
- External contributors see `ProprietaryFiles/` as an inaccessible submodule — don't ask them to modify it.

### Documentation gaps (issue #137)
- RTD `doc/BUILD.md` and `doc/CarlaDoc.md` are orphaned from the TOC — they build but aren't linked in `index.rst`.
- Python CommonLib (`ConfigHelper.py`, `MsgHelper.py`, `SocketHelper.py`) has zero docstrings; Sphinx autodoc will produce empty pages until added.
- The docs build clean with warnings as errors (RTD `fail_on_warning` + the `docs.yml` PR check); how versions are published is in `doc/DEVELOPER_GUIDE.md` → Documentation.
- Still open: the tutorials (`doc/tutorials/`) are stale and should be rebuilt on `tests/Python/SimpleEchoClient`.
- Python CommonLib (`ConfigHelper.py`, `MsgHelper.py`, `SocketHelper.py`) has almost no docstrings, and `doc/api/index.md` is still a stub; autodoc waits on the docstrings.
- C++ headers (`ConfigHelper.h`, `SocketHelper.h`, `MsgHelper.h`) have no Doxygen comments; C++ API doc is deferred until issue #137 is addressed.

### Build system
Expand Down
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Flexible Interface for XIL Simulation (FIXS)

<!-- rtd-about-start: doc/about.md includes from here to rtd-about-end -->
## FIXS Contacts
**Primary Maintainer**: University of Georgia\
**Primary Maintainer**: University of Georgia

Oak Ridge National Lab PIs
Max Chen\
Expand All @@ -11,12 +12,14 @@ zhoua@ornl.gov


## About
<!-- rtd-intro-start: doc/index.rst includes from here to rtd-intro-end -->
This interface is part of the **Real-Sim** project funded by Department of Energy (DOE)-Vehicle Technology Office (VTO)-Energy Efficient Mobility Systems (EEMS) program. It is part of the core tools of EEMS.

Real-Sim develops a multi-resolution X-in-the-loop (XIL) simulation framework to support inclusive testing and evaluation of emerging technologies such as connected and automated vehicles. The Real-Sim framework connects different simulation components and integrates various traffic and vehicle simulation tools, virtual environments, and XIL systems through a Flexible Interface for XIL Simulation (FIXS). Connections among different tools are transparent to users and handled by the interface behind the scenes. Co-simulation of various vehicle and traffic simulation tools with different XIL systems can be easily achieved and become a transparent "plug-and-play" process to users.

**Real-Sim Project Lead**: Oak Ridge National Lab\
**Real-Sim Project Team**: University of Georgia, University of Texas-Dallas
<!-- rtd-intro-end -->

**Links to current project presentation**\
https://www1.eere.energy.gov/vehiclesandfuels/downloads/2024_AMR/EEMS101_Chen_2024_o.pdf \
Expand All @@ -28,6 +31,7 @@ https://www1.eere.energy.gov/vehiclesandfuels/downloads/2022_AMR/eems067_deter_2
https://www.energy.gov/sites/default/files/2021-06/eems101_deter_2021_o_5-19_645pm_LR_ML.pdf

Publication list is upcoming...
<!-- rtd-about-end -->

# Other links
This README contains general information of the interface. For specific documentation of different simulators, check the following links:
Expand Down Expand Up @@ -131,8 +135,8 @@ scripts\dispatch\2_core_components.bat
# VISSIM components only
scripts\dispatch\3_vissim_components.bat

# CarMaker components only (auto-generates BuildConfig files)
powershell -ExecutionPolicy Bypass -File scripts\dispatch\4a_carmaker_components.ps1
# CarMaker components only
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5a_carmaker_components.ps1
```

For complete build system documentation including Debug/Release configurations, troubleshooting, and architecture details, see **[doc/BUILD.md](doc/BUILD.md)**.
Expand Down
31 changes: 18 additions & 13 deletions doc/BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,20 +209,20 @@ The release build process executes the following steps:
- `DriverModel_RealSim_legacy.dll` - VISSIM driver model, legacy/frozen (long API, VISSIM ≤ 2020)
- Output: `ProprietaryFiles/VISSIMserver/` + copied to `build/`

4. **CarMaker Components** (`4a_carmaker_components.ps1`)
4. **CarMaker Components** (`5a_carmaker_components.ps1`)
- Generates BuildConfig Python files for each CarMaker version
- `CarMaker.win64.exe` - CarMaker Office executables
- `libcarmaker4sl.mexw64` - CarMaker for Simulink MEX files
- Built for each version in `dependencies.yaml` (e.g., CM11, CM13)
- Output: `CarMaker/CM*/` + copied to `build/CarMaker/`

5. **dSPACE Libraries** (`4b_carmaker_dspace.ps1`) - Optional
5. **dSPACE Libraries** (`5b_carmaker_dspace.ps1`) - Optional
- `libRealSimDsLib_2024a_CM*.a` - Version-specific dSPACE libraries
- Only built if dSPACE ConfigurationDesk is detected
- Built for each CarMaker version
- Output: `CommonLib/` + copied to `build/CarMaker/`

6. **MEX RealSimSocket** (`5_mex_realsim_socket.ps1`) - Optional
6. **MEX RealSimSocket** (`6_mex_realsim_socket.ps1`) - Optional
- `RealSimSocket.mexw64` - MATLAB interface for socket communication
- Only built if MATLAB is detected
- Output: `CommonLib/` + copied to `build/`
Expand Down Expand Up @@ -265,14 +265,19 @@ FIXS releases are produced by a GitHub Actions pipeline (`.github/workflows/rele
- **Public source → built on the hosted runner every push** (`windows-2022`, matching the VS 2022 generator; `windows-latest` ships VS 2026 and is incompatible): yaml-cpp, `TrafficLayer.exe`, `CoordMerge.exe`, `VirtualEnvironment.lib` (SDK-free on the 0.9.0 train, #174), and all `Carla/` + `CommonLib` Python. No private submodule, no token.
- **Licensed-toolchain source → built manually on a licensed workstation**: the VISSIM DriverModel DLLs, CarMaker executables + CM4SL MEX, dSPACE libraries, and the MATLAB MEX. These need installed proprietary toolchains, so the hosted runner cannot build them. They ship as a prebuilt **bundle** that CI overlays into the build.

The hosted path is made proprietary-aware by the `RS_FIXS_AUTOMATION` environment flag, which tells `dispatch.bat` to skip the licensed steps (3 / 4a / 4b / 5) and consume the downloaded bundle instead.
The hosted path is made proprietary-aware by the `RS_FIXS_AUTOMATION` environment flag, which tells `dispatch.bat` to skip the licensed steps (VISSIM 3 / CarMaker 5a / dSPACE 5b / MEX 6) and consume the downloaded bundle instead.

### Rolling release channels

| Branch | Rolling prerelease | Notes |
|--------|--------------------|-------|
| `main` | `latest` | current train |
| `dev_v0.9.0` | `v0.9.0-alpha` | 0.9.0 train (SDK-free VirtualEnvironment, #174) |
| `main` | `stable` | release tier |
| `beta_vX.Y.Z` | `vX.Y.Z-beta` | test tier |
| `dev_vX.Y.Z` | `vX.Y.Z-alpha` | dev tier (e.g. `dev_v0.9.0` → `v0.9.0-alpha`, SDK-free VirtualEnvironment, #174) |

`scripts/dispatch/release_channel.ps1` owns this mapping. Each channel also has a
Read the Docs version under the same name (see the Developer Guide's documentation
section).

On every push to a release branch the pipeline builds the public core, overlays the matching proprietary bundle, packs one canonical zip, and (re)publishes the rolling prerelease anchored to that commit. Pull requests build + package only (no publish), and upload the zip as a workflow artifact for inspection.

Expand Down Expand Up @@ -342,7 +347,7 @@ Builds VISSIM driver model DLLs. Run this when modifying VISSIM interface code.

#### 4. CarMaker Components Only
```batch
powershell -ExecutionPolicy Bypass -File scripts\dispatch\4a_carmaker_components.ps1
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5a_carmaker_components.ps1
```
Builds CarMaker executables. Automatically:
- Reads CarMaker versions from `dependencies.yaml`
Expand All @@ -352,13 +357,13 @@ Builds CarMaker executables. Automatically:

#### 5. dSPACE Libraries Only
```batch
powershell -ExecutionPolicy Bypass -File scripts\dispatch\4b_carmaker_dspace.ps1
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5b_carmaker_dspace.ps1
```
Builds dSPACE libraries for CarMaker HIL integration. Run this when modifying dSPACE interface code.

#### 6. MEX RealSimSocket Only
```batch
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5_mex_realsim_socket.ps1
powershell -ExecutionPolicy Bypass -File scripts\dispatch\6_mex_realsim_socket.ps1
```
Builds MATLAB MEX file for socket communication. Run this when modifying MATLAB interface code.

Expand All @@ -376,7 +381,7 @@ scripts\dispatch\3_vissim_components.bat

**Scenario 3: Modified CarMaker User.c or integration**
```batch
powershell -ExecutionPolicy Bypass -File scripts\dispatch\4a_carmaker_components.ps1
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5a_carmaker_components.ps1
```

**Scenario 4: Testing full release package**
Expand Down Expand Up @@ -506,7 +511,7 @@ The `detect_tool_paths.ps1` script automatically locates installed tools:
CarMaker builds require a BuildConfig Python file that specifies compiler settings, include paths, and library dependencies. The build system automatically generates these:

**Process:**
1. `4a_carmaker_components.ps1` reads `dependencies.yaml`
1. `5b_carmaker_dspace.ps1` reads `dependencies.yaml`
2. For each CarMaker version (e.g., 11.1.2, 13.1.3):
- Detects MATLAB version
- Locates CarMaker installation
Expand All @@ -527,7 +532,7 @@ If you need custom BuildConfig settings, you can:
2. Manually edit the generated Python file
3. Re-run CarMaker build

Note: Re-running `4a_carmaker_components.ps1` will regenerate and overwrite custom changes.
Note: Re-running `5b_carmaker_dspace.ps1` will regenerate and overwrite custom changes.

## Troubleshooting

Expand Down Expand Up @@ -571,7 +576,7 @@ Error: Could not find MATLAB installation
- Ensure MATLAB is installed
- Update `dependencies.yaml` with correct MATLAB version
- Check MATLAB registry entries exist
- Try specifying MATLAB path manually in `5_mex_realsim_socket.ps1`
- Try specifying MATLAB path manually in `6_mex_realsim_socket.ps1`

---

Expand Down
2 changes: 1 addition & 1 deletion doc/CARLAdoc.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ CarlaSetup:
# whether or not enable Carla (default: false)
EnableCosimulation: true

EnableVerboseLog: false
EnableVerboseLog: false
# whether or not enable external control
# if set to true, the sumo vehicles will be updated according to the carla vehicles' state
# if set to false, the carla will be only visualizing the sumo vehicles
Expand Down
10 changes: 5 additions & 5 deletions doc/CarMakerDoc.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Please start with reading through the documentation, and playwith included Examp
- VirtrualEnvironment.lib

To simplify CarMaker project build process, we will build a library called VirtrualEnvironment.lib that contains all the
required helper functions. Open [VirtualEnvironment.sln](..%2FVirtualEnvironment%2FVirtualEnvironment.sln)
required helper functions. Open [VirtualEnvironment.sln](../VirtualEnvironment/VirtualEnvironment.sln)
inside the VirtualEnvironment folder using Visual Studio 2022. in Visual Studio 2022, find the solution explorer window on the right of the screen, right click the _VirtualEnvironment_, select _Properties_,
Then in the Property Page, find C/C++ -> General -> Additional Include Directories. Edit the carmaker version if necessary. And make sure yaml-cpp are included:
![VirtualEvn_cppsetup.png](img/VirtualEvn_cppsetup.png)
Expand Down Expand Up @@ -280,15 +280,15 @@ Using either relative path to the working directory or absolute path are OK
### Setup CM dSPACE

#### Obtain dSPACE library
RealSim comes with precompiled dSPACE library file that can be used directly under \CommonLib. If plan to use these libraries directly, can skip the next section and go to [this section](#prepare-for-dspace-build-configuration-desk) directly. Otherwise, follow the next section to compile customized dSPACE library.
RealSim comes with precompiled dSPACE library file that can be used directly under \CommonLib. If plan to use these libraries directly, can skip the next section and go to [this section](#prepare-for-dspace-build-configurationdesk) directly. Otherwise, follow the next section to compile customized dSPACE library.

#### Compile dSPACE library

**Option 1: Automated Build (Recommended)**

The dispatch system automatically builds dSPACE libraries for all CarMaker versions when dSPACE is detected:
```batch
powershell -ExecutionPolicy Bypass -File scripts\dispatch\4b_carmaker_dspace.ps1
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5b_carmaker_dspace.ps1
```

This script:
Expand All @@ -304,7 +304,7 @@ Tips: You can copy the `DsBuildLibrary.mk` into the CommonLib Folder and rename
_Refer to the following dSPACE documentation https://www.dspace.com/en/inc/home/support/kb/faqs/faq012.cfm._

After successfully execute the `buildRS_XXXX.bat` file, you should have something appear on command line window like
```commandline
```text
...
C:\Program Files\Common Files\dSPACE\CFD Compiler 24.1\target\x86_64-linux-gnu\bin\x86_64-linux-gnu-ar.exe: creating libRealSimDsLib_2024a_CM13_1_3.a
a - SocketHelper.o64
Expand Down Expand Up @@ -354,7 +354,7 @@ Note: currently, it is only for SCALEXIO and configuration desk dSPACE implement

The dispatch system automatically generates CarMaker BuildConfig Python files with correct dSPACE settings:
```batch
powershell -ExecutionPolicy Bypass -File scripts\dispatch\4a_carmaker_components.ps1
powershell -ExecutionPolicy Bypass -File scripts\dispatch\5a_carmaker_components.ps1
```

This generates `RS_CM{major}_{minor}_{patch}_BuildConfig_{matlab}.py` files with:
Expand Down
2 changes: 1 addition & 1 deletion doc/Carla_Linux_building.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ This compiles the Carla UE4 plugin and the `CarlaUE4Editor` target (long the fir

---

#### Issue 7 — `Permission denied` creating content directories
### Issue 7 — `Permission denied` creating content directories

The build fails late with:

Expand Down
4 changes: 1 addition & 3 deletions doc/Carla_Windows_building.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## CARLA 0.9.15 Windows Build Guide
# CARLA 0.9.15 Windows Build Guide

This document provides a practical guide for building CARLA 0.9.15 on Windows.

Expand Down Expand Up @@ -235,5 +235,3 @@ Ensure the following packages are installed specifically for Python 3.10, not ju
```
py -3.10 -m pip install setuptools wheel "numpy<2.0.0"
```

---
Loading
Loading