This project focuses on recognizing handwritten text using two different models: one for word recognition and another for word detection. The models are trained on the IAM dataset.
- Backend: A FastAPI backend (
main.py) that handles model inference. - Frontend: A React.js application for the user interface with firebase as backend.
- Models:
- Word Recognition Model: A CNN+LSTM+CTC architecture for recognizing words from images.
- Word Detection Model: A YOLOv8 model for detecting word boundaries in images.
- Detects word boundaries in handwritten text images using a YOLOv8 model.
- Recognizes words within the detected boundaries using a CNN+LSTM+CTC model.
The models are trained on the IAM Handwriting Dataset, which contains handwritten text lines and corresponding ground truth annotations.
- Python
- Node.js (for frontend)
- pip (Python package manager)
-
Clone the repository:
git clone git remote add origin https://github.com/nirajan1111/Handwritten_Text_Recognition_OCR.git cd Handwritten_Text_Recognition_OCR -
Set up a virtual environment and install the dependencies:
python3 -m venv venv source venv/bin/activate # On Windows, use `venv\Scripts\activate` pip install -r requirements.txt
-
Run the FastAPI backend:
uvicorn main:app --reload
The backend will be available at
http://127.0.0.1:8000.
-
Navigate to the
frontendfolder:cd frontend -
Install frontend dependencies:
npm install
-
Run the React.js development server:
npm start
The frontend will be available at
http://localhost:3000.
-
Word Recognition (CNN+LSTM+CTC):
- The model is trained on the IAM dataset and is responsible for recognizing words from handwritten images.
-
Word Detection (YOLOv8):
- This model detects word boundaries and outputs bounding boxes for each word in the input image.
- IAM Dataset: The models are trained using the IAM Handwriting Dataset.
- YOLOv8: Used for word boundary detection.
- CNN+LSTM+CTC: Architecture for word recognition.