GLM-OCR CDK

Deploy GLM-OCR on AWS with a one-command CDK stack. Includes a web UI, GPU start/stop controls, and a serverless API.

Architecture

User → CloudFront → S3 (frontend)
                  → API Gateway → Lambda (backend API)
                                    ↓
                              EC2 GPU (vLLM + GLM-OCR)
                                    ↓
                              S3 (uploads/results) + DynamoDB (tasks)

Service	Purpose
CloudFront	CDN for frontend + API routing
S3	Frontend hosting + document/result storage
API Gateway	HTTP API
Lambda	Backend API (FastAPI + Mangum) — task management, GPU control
EC2 (GPU)	vLLM serving GLM-OCR for inference
DynamoDB	Task tracking with TTL

Features

• 🔍 OCR — Upload images or PDFs, get structured Markdown/text output
• 🖥️ GPU Start/Stop — Control the GPU instance from the web UI to manage costs
• 📄 PDF Support — Multi-page PDF → per-page OCR with PyMuPDF
• 📋 Task History — View, retry, and delete past tasks
• 📦 Export — Download results as Markdown
• ⚡ Fast — vLLM serves GLM-OCR (0.9B params) with high throughput
• 🔒 Secure — GPU instance in private subnet, no public IP, accessed via VPC only

⚠️ Security Notice

This project deploys a publicly accessible web UI and API with no authentication. Anyone with the CloudFront URL can:

• Start/stop the GPU instance (incurring costs)
• Upload documents and run OCR
• View and delete task history

This is intended for personal use and demos. If you need to restrict access, consider:

• Adding CloudFront signed URLs or WAF IP restrictions
• Adding API Gateway authorizers (API key, Cognito, or IAM)
• Placing the CloudFront distribution behind a VPN

Prerequisites

• Node.js 18+
• AWS CDK CLI: npm install -g aws-cdk
• Docker (for building the Lambda container)
• AWS account with GPU instance quota (default: g6.2xlarge) in your target region

Quick Start

# Clone
git clone https://github.com/imers/glm-ocr-cdk.git
cd glm-ocr-cdk

# Install dependencies
npm install

# Bootstrap CDK (first time only)
cdk bootstrap

# Deploy
cdk deploy

After deployment, CDK outputs the Frontend URL (CloudFront). Open it in your browser.

Usage

1. Open the Frontend URL in your browser
2. Click ▶ Start GPU — the EC2 instance boots and loads the model (~3–5 min)
3. Upload an image or PDF
4. Click 🚀 Start OCR
5. View results, copy, or export
6. Click ⏹ Stop GPU when done to save costs

Cost Estimate

Resource	Approx. Cost
EC2 g6.2xlarge	~$0.98/hr (on-demand, only when running)
Lambda	Free tier covers most usage
NAT Gateway	~$0.045/hr + data
S3 + DynamoDB	Negligible
CloudFront	Negligible

Tip: Stop the GPU when not in use. The web UI makes this one click.

Configuration

Edit bin/glm-ocr-cdk.ts or pass CDK context variables:

# Use a different GPU instance type
cdk deploy -c gpuInstanceType=g5.xlarge

Available context variables (via -c key=value):

Variable	Default	Description
gpuInstanceType	g6.2xlarge	EC2 instance type (must have NVIDIA GPU)
amiId	(auto-detected)	Override the Deep Learning AMI ID for your region
frontendBuildPath	./frontend	Path to frontend assets

Available stack props (in bin/glm-ocr-cdk.ts):

Prop	Default	Description
lambdaMemorySize	1024	Lambda memory in MB
lambdaTimeoutSeconds	900	Lambda timeout

Note: The GPU AMI is automatically resolved to the latest Deep Learning Base OSS Nvidia Driver GPU AMI (Ubuntu 24.04) in your target region. Use -c amiId=ami-xxx to override if needed.

Project Structure

glm-ocr-cdk/
├── bin/                    # CDK app entry point
│   └── glm-ocr-cdk.ts
├── lib/                    # CDK stack definition
│   └── glm-ocr-stack.ts
├── docker/backend/         # Lambda container (FastAPI + Mangum)
│   ├── Dockerfile
│   ├── lambda_handler.py
│   ├── requirements.txt
│   └── app/
│       └── main_lambda.py
├── frontend/               # Static frontend (single HTML file)
│   └── index.html
├── ssm-restart.json        # SSM document to restart vLLM on the GPU instance
├── cdk.json
├── package.json
└── tsconfig.json

API Endpoints

Method	Path	Description
GET	/health	Health check
GET	/api/v1/gpu/status	GPU instance status + vLLM readiness
GET	/api/v1/gpu/logs	GPU instance logs (via SSM)
POST	/api/v1/gpu/start	Start GPU instance
POST	/api/v1/gpu/stop	Stop GPU instance
POST	/api/v1/tasks	Create OCR task (multipart file upload)
GET	/api/v1/tasks	List tasks
GET	/api/v1/tasks/:id	Get task status
GET	/api/v1/tasks/:id/result	Get task result
GET	/api/v1/tasks/:id/images	Get task page images
DELETE	/api/v1/tasks/:id	Delete task

Troubleshooting

vLLM not starting

Check logs via the web UI (Server Logs panel) or SSM:

aws ssm send-command \
  --instance-ids <INSTANCE_ID> \
  --document-name AWS-RunShellScript \
  --parameters '{"commands":["tail -50 /var/log/vllm.log"]}'

Or use the included restart script:

aws ssm send-command \
  --instance-ids <INSTANCE_ID> \
  --document-name AWS-RunShellScript \
  --parameters file://ssm-restart.json

GPU quota

If deployment fails with a capacity error, request a quota increase for the GPU instance type in your region via the AWS Service Quotas console.

Cleanup

cdk destroy

This removes all AWS resources including the EC2 instance, S3 buckets, and DynamoDB table.

License

MIT