Serverless Event Ingestion API

A production-style serverless API for ingesting JSON events over HTTPS and storing them reliably at scale.
Built using AWS-managed services to minimize operational overhead while maintaining scalability and reliability.

This project focuses on real-world cloud engineering practices rather than demo-style implementations.

---

Overview

The Serverless Event Ingestion API exposes a POST /events endpoint that accepts arbitrary JSON payloads such as user actions, logs, or application events.

Each incoming event is:

• Validated
• Enriched with metadata (unique ID and timestamp)
• Persisted to DynamoDB for downstream processing or analytics

The system is fully serverless and scales automatically with demand.

---

Architecture

Core components used in this project:

• API Gateway (HTTP API)
  Exposes a public HTTPS endpoint for event ingestion.

• AWS Lambda (Go)
  Handles request validation, processing, and persistence logic.

• DynamoDB
  Stores ingested events with a flexible schema.

• Terraform
  Used for Infrastructure as Code to provision Lambda, IAM roles, and DynamoDB resources.

---

API Specification

Endpoint

POST /events

Headers

Content-Type: application/json

Example Request

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"user":"aditya","action":"login"}' \
  https://<api-id>.execute-api.us-east-1.amazonaws.com/events

Example Response

{
  "status": "ok",
  "id": "b2f4e303-41af-4bbd-bac1-b3fbce46c9ae"
}

---

Event Processing Flow

1. API Gateway receives the HTTP request
2. Lambda validates the HTTP method and parses the JSON body
3. A unique event ID and ingestion timestamp are added
4. The event is stored in DynamoDB
5. A success response containing the event ID is returned

---

Project Structure

.
├── cmd/handler
│   └── main.go        # Go Lambda handler
├── main.tf            # Terraform infrastructure definition
├── go.mod
├── go.sum
└── README.md

---

Key Technical Learnings

• Differences between API Gateway REST APIs (v1) and HTTP APIs (v2)
• Building statically linked Go binaries for AWS Lambda
• Designing serverless ingestion systems with minimal operational overhead
• Debugging real-world cloud issues involving IAM, Terraform state, and runtime compatibility
• Using Terraform to manage AWS infrastructure declaratively

---

Why Serverless?

• Automatic scaling with traffic
• No server management
• Pay-per-use cost model
• High availability by default

This architecture is well-suited for event ingestion, telemetry pipelines, and backend integrations.

---

Possible Enhancements

• Event schema validation
• Authentication and authorization
• DynamoDB Streams for downstream processing
• Dead-letter queues for failed writes
• Metrics and observability integration

---

Status

This project is fully functional and deployed on AWS.
It was built as a hands-on cloud engineering exercise focused on realistic infrastructure design and debugging.