Skip to content

About

Handwritten answer-sheet grader: YOLO text segmentation feeding an LLM semantic scorer, reaching 9% character error rate.

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

5 Commits

Folders and files

Repository files navigation

Handwritten Text Recognition

This project focuses on recognizing handwritten text using two different models: one for word recognition and another for word detection. The models are trained on the IAM dataset.

Project Structure

  • Backend: A FastAPI backend (main.py) that handles model inference.
  • Frontend: A React.js application for the user interface with firebase as backend.
  • Models:
    1. Word Recognition Model: A CNN+LSTM+CTC architecture for recognizing words from images.
    2. Word Detection Model: A YOLOv8 model for detecting word boundaries in images.

Features

  • Detects word boundaries in handwritten text images using a YOLOv8 model.
  • Recognizes words within the detected boundaries using a CNN+LSTM+CTC model.

Dataset

The models are trained on the IAM Handwriting Dataset, which contains handwritten text lines and corresponding ground truth annotations.

Installation

Prerequisites

  • Python
  • Node.js (for frontend)
  • pip (Python package manager)

Backend Setup

  1. Clone the repository:

    git clone git remote add origin https://github.com/nirajan1111/Handwritten_Text_Recognition_OCR.git
    
    cd Handwritten_Text_Recognition_OCR
  2. Set up a virtual environment and install the dependencies:

    python3 -m venv venv
    source venv/bin/activate  # On Windows, use `venv\Scripts\activate`
    pip install -r requirements.txt
  3. Run the FastAPI backend:

    uvicorn main:app --reload

    The backend will be available at http://127.0.0.1:8000.

Frontend Setup

  1. Navigate to the frontend folder:

    cd frontend
  2. Install frontend dependencies:

    npm install
  3. Run the React.js development server:

    npm start

    The frontend will be available at http://localhost:3000.

Models

  • Word Recognition (CNN+LSTM+CTC):

    • The model is trained on the IAM dataset and is responsible for recognizing words from handwritten images.
  • Word Detection (YOLOv8):

    • This model detects word boundaries and outputs bounding boxes for each word in the input image.

Acknowledgments

  • IAM Dataset: The models are trained using the IAM Handwriting Dataset.
  • YOLOv8: Used for word boundary detection.
  • CNN+LSTM+CTC: Architecture for word recognition.

About

Handwritten answer-sheet grader: YOLO text segmentation feeding an LLM semantic scorer, reaching 9% character error rate.

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages