Repository navigation
Replay MOOS-IvP missions recorded under pHelmIvP against the helm library - #5
Merged
tsaubergine merged 8 commits intoNov 4, 2026
Conversation
…rary - src/test/missions/record.sh builds all of MOOS-IvP from the submodule, runs each mission in missions.txt headless with time warp, and keeps the vehicle's helm inputs and outputs as a fixture. A uTimerScript added to the mission makes the pokes, since uPokeDB's posts reach pHelmIvP seconds after their timestamps under warp. A first run finds pHelmIvP's variables and the second logs each of them explicitly, since pLogger's wildcard logging misses a variable's first posts. - replay feeds the recorded inputs to ivpico::helm::Helm at pHelmIvP's iteration times and compares state, allstop, modes, initialize, behavior postings, life events, active behaviors and DESIRED_*. - The "Record missions" workflow runs record.sh and uploads the fixtures. - init.sh installs xz-utils for the compressed fixtures. First fixture: s1_alpha, which matches on every stream, and on DESIRED_HEADING for 1463 of 1464 iterations (the other is 2 degrees off). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VLcYK4e29B5FiPbDDJy5fp
pHelmIvP posts its initialize variables before it registers for anything, so a value posted before the helm started (e.g. DEPLOY=true) is replaced by the helm's own initialize value. The helm library applied queued inputs after initialize instead. record.sh now launches pLogger right after MOOSDB, so the log has pHelmIvP's startup, starts the pokes once pHelmIvP is up, finds the log directories wherever the mission's pLogger writes them, and warns if a log misses pHelmIvP's startup. replay names fixture directories given with a trailing slash. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VLcYK4e29B5FiPbDDJy5fp
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VLcYK4e29B5FiPbDDJy5fp
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VLcYK4e29B5FiPbDDJy5fp
pHelmIvP's initialize posts come back from the MOOSDB like its other posts. It doesn't store them again, but it still acts on them, so "initialize MOOS_MANUAL_OVERRIDE = false" (s5_echo) starts the helm in DRIVE. They, and the other startup posts, arrive before pHelmIvP's first iteration, so the helm library now feeds them back on its first one. replay: - mail latency 5 ms (best across the recorded missions); - a different value on the same iteration counts as one mismatch, not two; - streams with 20 or more changes may differ on up to 5% of them, since continuous values (distances, ETAs, track points) differ whenever pHelmIvP's navigation arrived slightly off its timestamp (DESIRED_* stays at 1%); - an optional second argument prints one stream in full. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VLcYK4e29B5FiPbDDJy5fp
- Drop the missions' WildCardOmitPattern lines, so BHV_STATUS and the like are logged and compared. - Run pAntler in its own process group, and kill the group if it hasn't exited 20 s after SIGINT (s7_golf's left MOOSDB and pLogger running). - The check for pHelmIvP's startup no longer trips on SIGPIPE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VLcYK4e29B5FiPbDDJy5fp
setsid may fork, leaving $! pointing at a process that already exited. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VLcYK4e29B5FiPbDDJy5fp
- s2_bravo, s3_charlie, s4_delta, s5_echo, s7_golf, s10_juliet and x1_legs join s1_alpha. Every stream matches in all eight, and DESIRED_* matches on all but 0-7 iterations of each. - s7_golf parks soon after deploying (MissingDecVars), and x1_legs is MALCONFIG (its .bhv uses BHV_LegRun parameters this MOOS-IvP rejects), in pHelmIvP as in IvPico. - replay reads pHelmIvP's configuration in reverse, as MOOS's mission reader hands it over, which sets the order of the domain variables (e.g. in "MissingDecVars:speed,course"). - record.sh copies the mission's .bhv files when pHelmIvP never started far enough to ask pLogger for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VLcYK4e29B5FiPbDDJy5fp
2 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #4 (helm library), and targets its branch. Merge #2, #3 and #4 first.
This is the second half of Phase 1: MOOS-IvP's own missions become IvPico's regression suite. Each mission runs once under real pHelmIvP and is recorded. The replay test then feeds the same inputs to
ivpico::helm::Helmat pHelmIvP's iteration times, and checks that it posts and decides the same things.src/test/missions/README.mdhas an overview diagram.Results
Eight single-vehicle missions from
ivp/missionsare recorded, about 3.2 MB in all, and all eight pass. Every compared stream matches, and DESIRED_* matches on all but a handful of iterations:Two of these don't run as a mission would:
.bhvuses BHV_LegRun parameters this MOOS-IvP version rejects.pHelmIvP does exactly the same in both, so they now test those paths. x1_legs could move to another mission later.
The replay found three real fidelity bugs, now fixed in the helm library
initialize MOOS_MANUAL_OVERRIDE = falsestarts it in DRIVE.These change
src/lib/helm/helm.cpp, which #4 adds. I can move them into #4 if you'd rather.Recording (
record.sh,missions.txt, "Record missions" workflow)record.shbuilds all of MOOS-IvP from the submodule intobuild/missions(build-moos.sh --minrobot,build-ivp.sh --nogui). It runs each mission with time warp 10 and keeps each vehicle's trimmed log (xz),.moosand.bhv. Getting reliable logs out of MOOS took a few adjustments to the mission files:@ 0.WildCardOmitPatterns are dropped so that BHV_STATUS is logged too.The "Record missions" workflow (
workflow_dispatch, with an optional list of missions) runsrecord.shon GitHub and uploads the fixtures as an artifact to commit. The recordings should be redone wheneverexternal/moos-ivpmoves.Replay (
replay.cpp, one ctest per fixture)Inputs: every logged post to a variable pHelmIvP registered, except pHelmIvP's own, goes in as a
Variableonce it is 5 ms old at an iteration. 5 ms matched best across the missions. pHelmIvP's own posts come back through the helm's own echo.Configuration:
HelmConfigcomes from the.moosfile's pHelmIvP block, read in reverse as MOOS's mission reader hands it over. That order sets the order of the domain variables.Compared as sequences of changes, within ±1 iteration:
Numbers inside values match within 0.1% (at least 0.01). A stream with 20 or more changes may differ on up to 5% of them, because pHelmIvP sometimes gets a nav update slightly before or after its timestamp, which shows up in distances, ETAs and track points.
DESIRED_* is compared iteration by iteration: within one domain step on at least 99% of iterations, and never off for more than 3 in a row.
On failure it prints the first mismatches in each stream.
replay <fixture> "<behavior> <VAR>"prints one stream in full.Also
init.shinstallsxz-utils, which the Ubuntu base images lack.Test plan
ctest:helm_libs,helmand the eightreplay_*tests pass (about 12 s).record.shin this sandbox (MOOSGeodesy without PROJ, since PROJ couldn't be downloaded here).main, or by selecting this branch).Generated by Claude Code