Smart Pothole Detection & Civic Management

eNivaran is an ML-powered road safety and civic issue reporting system designed to help create safer, smarter cities through community participation and cutting-edge technology. Users can report civic issues like potholes, view existing complaints, and track their status, while administrators can manage and update these reports. The system utilizes advanced AI models for pothole detection and duplicate prevention, along with real-time communication features.

Core Functionalities

User Features

• Civic Issue Reporting: Users can submit detailed complaints including:

  • Text description and issue categorization
  • Location (automatically converted to coordinates using Geopy)
  • Image evidence
  • Real-time status tracking
  • Two-way communication with administrators
  • Speech-to-Text Integration:
    • Seamless voice input for complaint descriptions using the Web Speech API.
    • Multi-language Support: Automatically translates spoken Hindi (and other Indic languages) to English using Argos Translate (offline) with Gemini AI as a robust fallback.

• Complaint Management:

  • Public View (/complaints): View all non-duplicate complaints, sortable by time or upvotes. Includes basic search by ID. Users can upvote complaints.
  • User View (/my_complaints): Logged-in users can view the status and details of complaints they have submitted.
  • Feedback System: Users can provide feedback and ratings for resolved complaints.

• AI-Powered Tools:

  • AI Pothole Detection: Upload an image to the 'Tools' section to automatically detect potholes using a pre-trained Pytorch model (pothole_detector_v1.onnx). The system analyzes depth, estimates severity, and provides road condition assessment with annotated visualizations.
  • AI Video Analysis: Upload a video of a road to the 'Tools' section to get a frame-by-frame analysis. The system uses the pothole_detector_v1.onnx model to:
    • Identify potholes in each frame.
    • Produce an annotated output video showing the detections in real-time.
  • Unified Pothole Detector: Advanced street view integration using KartaView imagery for comprehensive road assessment.
  • Admin Pothole Inspector: Dedicated tool for administrators to fetch and analyze street-level images from specific addresses.

• Real-time Communication:

  • Firebase-powered chat system between users and administrators
  • File sharing capabilities in chat
  • Unread message notifications
  • Chat history persistence
  • New chat notifications

• AI Assistance:

  • Google Gemini-powered chatbot (J.A.R.V.I.S)
  • Context-aware responses
  • File upload support
  • Markdown-formatted responses
  • Chat history management

• Leaderboard System: Track user engagement and contributions through a comprehensive leaderboard.

Administrative Features

• Higher Admin Dashboard (/admin_dashboard): Comprehensive administrative interface with:

  • Comprehensive Complaint Management: View all complaints with user details, submission times, and current status
  • Status and Remarks Updates: Ability to update complaint status (Submitted, In Progress, Resolved, etc.) and add detailed remarks
  • Complaint Deletion: Authority to delete irrelevant, resolved, or spam complaints
  • Advanced Filtering: Filter complaints by ID, city, or status for efficient management
  • Real-time Communication: Direct chat system with users for complaint-specific discussions
  • Feedback Management: Handle user feedback and ratings for resolved complaints

• Advanced Duplicate Prevention: Multi-modal duplicate detection system that:

  • Analyzes text similarity using Sentence-BERT embeddings
  • Compares image features using ResNet50
  • Checks location proximity with geospatial clustering
  • Uses XGBoost to classify potential duplicates with confidence scores

• User Authentication & Security:

  • Secure signup/login system with password hashing
  • Google OAuth 2.0 Integration for one-click login
  • Email Verification system for user validity
  • Session-based authentication
  • Role-based access control (user/admin)
  • Secure file upload handling
  • Request validation and sanitization

• Analytics & Statistics:

  • Real-time pothole statistics
  • Severity categorization (high, medium, low)
  • Road condition assessment
  • Complaint resolution tracking
  • User engagement metrics

Technology Stack

Frontend Architecture

• Hybrid Approach:
  • Server-Side Rendering: Flask + Jinja2 Templates for core structure and SEO.
  • Client-Side Interactivity: React.js (via CDN with in-browser Babel) for dynamic components.
• UI Framework: Bootstrap 5 for responsive layout and styling.
• Styling: Custom CSS + Bootstrap Icons + AOS (Animate On Scroll).
• Speech Recognition: Web Speech API for real-time voice input.

Backend Systems

• Core Framework: Python 3.11+ with Flask.
• Authentication:
  • Session-based auth with Werkzeug security.
  • Google OAuth 2.0 verification and login.
  • Email verification via Flask-Mail.
• Database:
  • SQLite: Primary relational database (Users, Complaints, Feedback).
  • Firebase Realtime Database: Low-latency chat synchronization.

Artificial Intelligence & Machine Learning

• Generative AI:
  • Google Gemini 2.0 Flash: Powered by google-genai SDK for the AI Chatbot (J.A.R.V.I.S) and Case Summarizer.
• Translation Engine:
  • Argos Translate: Offline, privacy-first translation for multi-lingual support.
  • Gemini AI Fallback: Ensuring reliability when offline models are unavailable.
• Computer Vision (Pothole Detection):
  • YOLOv8 (Ultralytics): State-of-the-art object detection.
  • ONNX Runtime: Optimized model inference (pothole_detector_v1.onnx).
  • OpenCV & Pillow: Image preprocessing and annotation.
• Duplicate Detection Engine:
  • Text: Sentence-Transformers (SBERT) for semantic similarity.
  • Images: ResNet50 (via PyTorch/Torchvision) for visual feature embedding.
  • Logic: XGBoost classifier for final duplicate probability.
  • Clustering: Scikit-learn (DBSCAN/KMeans) for geospatial grouping.

Services & Integrations

• Maps & Location: Geopy (Nominatim) for address-to-coordinate conversion.
• Street View: KartaView API for fetching street-level imagery.
• Email: SMTP integration for notifications.

DevOps & Tools

• Containerization: Docker & Docker Compose.
• Environment Management: python-dotenv.
• Version Control: Git.

Environment Setup

Option 1: Docker Deployment (Recommended)

1. Prerequisites:

   • Docker and Docker Compose installed on your system
   • At least 4GB RAM available for the container

2. Environment Configuration: Ensure you have a .env file in the root directory with:

   GEMINI_API_KEY=your_gemini_api_key
   FLASK_SECRET_KEY=your_secret_key
   FIREBASE_DATABASE_URL=your_firebase_url
   

3. Firebase Setup:

   • Place your firebase-service-account.json in the root directory
   • Enable Realtime Database in your Firebase console
   • Configure database rules for security

4. Quick Start with Docker:

   # Make the deployment script executable (Linux/macOS)
   chmod +x docker-run.sh
   
   # Run the deployment script
   ./docker-run.sh

   Or manually:

   # Build and start the container
   docker-compose up --build -d
   
   # View logs
   docker-compose logs -f

5. Access the Application:

   • Web Application: http://localhost:5000
   • Admin Login: username admin001, password admin$001

6. Docker Management:

   # Stop the application
   docker-compose down
   
   # Restart the application
   docker-compose restart
   
   # View container status
   docker-compose ps
   
   # Clean up (removes containers and volumes)
   docker-compose down -v

Option 2: Local Development Setup

1. Clone the Repository:

   git clone https://github.com/Souravshukla007/eNivaran_Codex2.0_Dynamite_PS07.git
   cd eNivaran_Codex2.0_Dynamite_PS07

2. Environment Configuration: Create a .env file in the root directory with:

   GEMINI_API_KEY=your_gemini_api_key
   FLASK_SECRET_KEY=your_secret_key
   FIREBASE_DATABASE_URL=your_firebase_url
   

3. Firebase Setup:

   • Place your firebase-service-account.json in the root directory
   • Enable Realtime Database in your Firebase console
   • Configure database rules for security

4. Virtual Environment:

   # For Windows
   python -m venv venv
   .\venv\Scripts\activate
   
   # For Linux/macOS
   python3 -m venv venv
   source venv/bin/activate

5. Install Dependencies:

   pip install -r requirements.txt

   • Install Translation Packs:

   python install_translation_packs.py

6. Initialize the Application:

   flask run
   # or
   python flask_app.py

   Access at http://127.0.0.1:5000/

7. Database Setup:

   • Automatic initialization of SQLite databases:
     • users.db
     • complaints.db
     • pothole_data.db
   • Tables created on first run

User Management & Roles

eNivaran implements a comprehensive role-based access control system with distinct user types and permissions:

User Roles

1. Citizen Users

• Registration: Users can register for a new account by providing a username, full name, and password. The password is securely stored using a password hash.
• Login: Registered users can log in to the platform to access its features.
• Permissions:
  • Submit civic complaints with images/videos
  • View public complaints and upvote them
  • Track status of their submitted complaints
  • Communicate with administrators via chat
  • Provide feedback on resolved complaints
  • Access AI-powered tools for pothole detection
  • View leaderboard rankings

2. Normal Administrators

• Login: Basic administrative login with predefined credentials (admin001/admin$001)
• Core Platform Access: Access to essential complaint management features
• Administrative Permissions:
  • View all complaints with user details and submission times
  • Update complaint status (Submitted, Approved, Rejected, On Hold, Completed)
  • Add remarks and feedback to complaints
  • Delete irrelevant, resolved, or spam complaints
  • Filter complaints by ID, city, or priority for efficient management
  • Direct communication with users via complaint-specific chat
  • View basic complaint statistics and status distribution

3. Higher Administrators

• Login: Advanced administrative login with elevated credentials (higher001/higher$001)
• Full Platform Access: Complete access to all administrative features plus advanced analytics
• Enhanced Administrative Permissions:
  • All Normal Admin permissions
  • Advanced Analytics Dashboard: Real-time charts and statistics including:
    • Complaints by status (pie chart)
    • Complaints over time (line chart)
    • Issue types distribution (stacked bar chart)
    • Top cities by complaints (bar chart)
    • User feedback sentiment analysis (donut chart)
  • AI Case Summarizer: Generate AI-powered summaries of complaint cases
  • User Feedback Management: View and analyze user feedback and ratings
  • Case Review System: Access completion details and proof images from field staff
  • Pothole Detector Access: Access to Admin Pothole Inspector and Unified Pothole Detector tools
  • Advanced Filtering: Enhanced search and filtering capabilities
  • Supervisory Oversight: Monitor and review actions taken by Normal Administrators

Security Features

• Secure Authentication: Password hashing and session-based authentication
• Role-Based Access Control: Strict separation between user and admin functionalities
• Data Validation: Comprehensive input validation and sanitization
• Secure File Handling: Safe file upload and storage mechanisms

System Access

User Access

1. Citizen User Registration:
   • Sign up at /signup
   • Required: username, full name, password
   • Automatic redirect to homepage
   • Access to all user features
   • Secure password storage with hashing

Administrative Access

1. Normal Administrator:

   • Username: admin001
   • Password: admin$001
   • Core complaint management features
   • Status updates and remarks
   • User communication via chat
   • Basic filtering and search capabilities
   • Access to essential administrative tools

2. Higher Administrator:

   • Username: higher001
   • Password: higher$001
   • Complete platform access including all Normal Admin features
   • Advanced analytics dashboard with real-time charts
   • AI case summarizer for complaint analysis
   • User feedback management and sentiment analysis
   • Case review system with completion details
   • Access to advanced pothole detection tools
   • Supervisory oversight capabilities
   • Enhanced filtering and search features

System Architecture

eNivaran follows a modern, scalable architecture with clear separation of concerns:

Frontend Architecture

• Single Page Application (SPA): Built with React for dynamic user interactions
• Component-Based Design: Modular components for maintainability and reusability
• Responsive Design: Bootstrap 5 ensures compatibility across devices
• Real-time Updates: WebSocket integration for live notifications and chat

Backend Architecture

• RESTful API: Flask-based backend with clean API endpoints
• Database Layer: SQLite for core data, Firebase for real-time features
• File Storage: Secure handling of images and videos with proper validation
• Authentication System: Session-based with secure password hashing

AI/ML Pipeline

• Model Serving: ONNX runtime for efficient model inference
• Image Processing: OpenCV for preprocessing and postprocessing
• Duplicate Detection: Multi-modal ML pipeline with feature extraction and classification
• Real-time Processing: Optimized for fast response times on user uploads

Communication Layer

• Firebase Integration: Real-time database for chat and notifications
• WebSocket Support: Live updates for complaint status and chat messages
• File Transfer: Secure file sharing within chat system

How It Works

1. User Authentication Flow

• Login/Signup validation with secure password hashing
• Session creation and management
• Role-based routing (user vs admin)
• Persistent authentication across browser sessions

2. Pothole Detection Process

• Image/video upload with validation and size limits
• ONNX/YOLO model inference for object detection
• Depth estimation and severity analysis
• Results storage with metadata
• Statistics update and visualization

3. Complaint Management Workflow

• Form submission with comprehensive validation
• Geocoding of address to coordinates using Geopy
• Multi-modal duplicate detection:
  • Text similarity analysis using Sentence-BERT
  • Image feature comparison using ResNet50
  • Location proximity check with geospatial clustering
• Storage with appropriate flags and metadata
• Real-time notifications to administrators

4. Communication System

• Firebase-powered real-time chat between users and admins
• File sharing capabilities within chat conversations
• Message history persistence and retrieval
• Read status tracking and delivery confirmations
• Unread message notifications with badges

5. Admin Operations

• Comprehensive complaint review interface
• Status updates with detailed remarks
• Advanced filtering and search capabilities
• Direct user communication via complaint-specific chat
• Feedback management and user rating system

6. AI Assistance System

• Google Gemini-powered chatbot (J.A.R.V.I.S)
• Context-aware responses based on conversation history
• File upload support for document analysis
• Markdown-formatted responses for rich content
• Persistent chat history management

7. Advanced Features

• Unified Pothole Detector: Integration with KartaView for street-level imagery analysis
• Admin Pothole Inspector: Dedicated tool for address-based road assessment
• Leaderboard System: User engagement tracking and gamification
• Real-time Analytics: Live statistics and performance metrics

---