SCENE-DETECT is a smart, privacy-conscious scene understanding system designed to assist blind and visually impaired individuals. It automatically detects what is happening in a room — specifically:
- Person is present and OK
- Person is present and NOT OK (e.g., collapsed or lying)
- No person in the scene
The system uses pose detection and intelligent rules to classify each situation and optionally provides a voice alert when something is wrong.
No real camera frame is ever shown - every output above is an abstract skeleton drawn purely from pose keypoints.
- Privacy first: no original images are stored or shown
- Real-time and image-based variants
- Works with single or multiple persons
- Powered by MediaPipe and YOLOv8-Pose
- Audio alerts for critical events
app.py is a Gradio front end that wraps the YOLOv8-Pose pipeline (core/detector.py)
into one interactive demo with three tabs:
- Image - upload a photo (or try a bundled sample) and get an instant skeleton + classification
- Video - upload a short clip and get back an annotated skeleton video with a summary
- Live Webcam - real-time, multi-person detection streamed straight from your browser, with a visual
alert banner and alert tone when someone is flagged "NOT OK" (no server-side audio is used, since a hosted
deployment has no speakers of its own - that also replaces the desktop scripts'
pyttsx3voice alert for the web)
Run it locally:
pip install -r requirements.txt
python app.pyThen open the local URL Gradio prints (defaults to http://127.0.0.1:7860).
Single Image Scene Detection
This script processes individual images from three folders:
/Nobody//Person OK//Person not OK/
Live Scene Detection (Single Person Only)
Uses a webcam or phone camera feed and processes each frame using MediaPipe Pose.
- Only detects one person at a time
- Very accurate pose quality (33 points, including z-depth)
- Draws stick-figure in real time on a clean canvas
- Announces via voice: "Person not okay!"
- Discards the original image after use (privacy maintained)
Live Scene Detection (Multi-Person)
Uses YOLOv8-Pose for real-time detection of multiple people.
- Draws each person as a stick figure in true position
- Classifies each person individually: OK / NOT OK
- Places labels near each person
- Voice alert if anyone is "NOT OK"
- All raw image data is deleted after use to preserve privacy
All live scripts support audio alerts via pyttsx3. The system says:
“Alert. Person not okay.”
...when a collapsed person is detected.
Web app only (app.py):
pip install -r requirements.txtDesktop scripts (image / MediaPipe / YOLO live detection, with voice alerts):
For Linux
pip install -r requirements_linux.txtFor Windows
pip install -r requirements_win.txtBasic requirements:
- Python 3.8+
- OpenCV
- MediaPipe
- Ultralytics (for YOLOv8)
- pyttsx3
- Gradio (web app only)
Web app (all-in-one image / video / live webcam demo):
python app.pyImage detection:
python detection_with_images.pyLive detection (MediaPipe):
python detection_with_live_mediapipe.pyLive detection (YOLOv8 multi-person):
python detection_with_live_yolo.pySCENE-DETECT ensures:
- Original RGB frames are deleted immediately after pose extraction
- Output is shown as skeleton-only canvas
- Designed with assistive and ethical AI principles in mind
- 🏠 Smart home monitoring for elderly or visually impaired
- 🧪 Human activity classification demos
- 👩🏫 Teaching pose estimation + privacy-centered design
- 📊 Dataset analysis and model evaluation
- Farrukh Rasool, Ammad Majid, Sai Pavan Bolagani, Syed Annas Jawed, Soham Shripad Kharadkar, Venkata Kishore Dannana
- Smart Scene Detect Project, OTH Amberg-Weiden
- Course: Industry 4.0 Project/ Artificial Intelligence for Industrial Applications
This project is licensed under the terms of the GNU General Public License v3.0 (GPLv3).
You are free to:
Share — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material for any purpose
Under the following terms:
Copyleft — If you modify and distribute the project, you must release it under the same GPLv3 license.