MarIA — Autonomous assistance robot for elderly care

Bachelor's thesis in Computer Engineering · Universidad Europea de Madrid · 2026

MarIA is a mobile robot that follows a person around the home, detects falls in real time and navigates while avoiding obstacles. Everything runs on board an NVIDIA Jetson Xavier AGX with a single Intel RealSense D435i camera.

What it does

• Person following — YOLOv8 (TensorRT) detects the person, the depth image gives the distance, and a proportional controller keeps the robot at a set distance. The target topic runs at ~15 Hz under full system load.
• Fall detection — YOLO11n-pose keypoints + geometric reasoning (not an end-to-end classifier). Classifies OK / FLOOR_HIP / FLOOR_BODY / KNEES / HIGH_DROP, confirmed over 5 consecutive frames. Runs at ~19 FPS on the Jetson GPU and serves a live dashboard via FastAPI.
• Search behaviour — a state machine switches between FOLLOWING and SEARCHING: if the person is lost, Nav2 drives to their last known position.
• Obstacle avoidance — nvblox builds a persistent 2D ESDF map (with the tracked person masked out) and the Nav2 Collision Monitor applies stop/slow zones matched to the robot's URDF.
• Odometry — wheel encoders from the Arduino firmware, optionally fused with an IMU through a robot_localization EKF.

Architecture

flowchart LR
    CAM[RealSense D435i<br/>color + depth] --> YOLO[YOLOv8 TensorRT<br/>person detection]
    CAM --> FD[FallDetector<br/>YOLO11n-pose + FastAPI]
    YOLO --> PD[person_distance_node]
    CAM --> PD
    PD --> SM[state machine<br/>FOLLOWING / SEARCHING]
    CAM --> MASK[obstacle_mask_node] --> NVB[nvblox ESDF]
    NVB --> NAV[Nav2]
    SM --> NAV
    SM --> CM[Collision Monitor]
    NAV --> CM
    CM --> BR[megapi_bridge_node] --> MCU[Arduino MegaPi<br/>motors + encoders]
    MCU -->|encoders| BR -->|/odom| NAV

Loading

Full description (in Spanish): docs/Arquitectura.md.

Hardware

Component	Role
NVIDIA Jetson Xavier AGX	Main compute (TensorRT, CUDA)
Intel RealSense D435i	Color for detection, depth for distance, obstacles and falls
Arduino MegaPi (Makeblock) + 2 DC motors with encoders	Actuation and wheel odometry
Phidgets IMU (optional)	Yaw rate for EKF fusion

Tech stack

ROS 2 Humble · Isaac ROS (YOLOv8 TensorRT, nvblox) · Nav2 · robot_localization · Python · FastAPI · Ultralytics YOLO11-pose · OpenCV · Arduino C++ · Docker

Repository structure

ros2_ws/src/
  yolov8_person_distance/   ROS 2 package: perception, control, state machine, launch files, URDF, configs
  image_tools_relay/        QoS bridge for the camera stream
  cmd_vel_to_serial/        Serial bridge (legacy)
falldetector/               Fall detection service (FastAPI + YOLO11-pose), with unit tests
firmware/arduino/           MegaPi firmware: non-blocking serial parser, 500 ms safety timeout, encoder output
scripts/                    Launchers, diagnostics and simulation test suite
docs/                       Architecture, obstacle avoidance and run commands (Spanish)

Running it

The system runs inside the Isaac ROS dev container on the Jetson. Step-by-step commands are in docs/Comandos.md. The FallDetector can also run standalone on a laptop with a webcam — see falldetector/README.md.

cd falldetector
pip install -r requirements.txt
python -m pytest tests/

Team

Two-person bachelor's thesis project.

• Víctor Martín Lorenzo — system design and most of the implementation: ROS 2 pipeline, person following, fall detection service, Nav2/nvblox navigation, odometry and EKF, final Arduino firmware and hardware integration. LinkedIn · GitHub
• Javier Rafael Cubas Alonso — RANSAC floor-plane estimation for fall detection, first version of the Arduino firmware, and the web monitoring dashboard.

Research prototype built for a bachelor's thesis. It is not a medical device and must not replace human supervision.