Analytics Platform

Real-time event analytics platform built with Go, ClickHouse, and Redpanda (Kafka). Collect, stream, store, and query analytics events at scale.

Architecture

Client -> Collector -> Redpanda -> Consumer -> ClickHouse -> Analytics API
         (HTTP)       (Kafka)      (batch)    (columnar)    (REST)

Services

Service	Role	Port
collector	Accepts analytics events via HTTP, validates, and pushes to Redpanda	:8080
consumer	Batches events from Redpanda and writes to ClickHouse	—
analytics-api	REST API for querying events, metrics, funnels, and user management	:8081
web dashboard	Next.js UI for visualizing analytics (optional)	:3000

Quick Start

docker compose up --build -d

Open http://localhost:3000, create an account, create a project, and start sending events.

Send Events

curl -X POST http://localhost:8080/collect \
  -H 'X-API-Key: demo_key' \
  -H 'Content-Type: application/json' \
  -d '{"event":"page_view","project_id":"demo_project","properties":{"url":"/home"}}'

Query Data

# Get recent events
curl http://localhost:8081/events?project_id=demo_project&limit=5

# Daily active users
curl http://localhost:8081/metrics/dau?project_id=demo_project

# Conversion funnel
curl -X POST http://localhost:8081/analytics/funnel \
  -H 'Content-Type: application/json' \
  -d '{"project_id":"demo_project","steps":["page_view","signup","purchase"]}'

Tech Stack

• Go — Collector, consumer, analytics API
• ClickHouse — Columnar analytics storage
• Redpanda — Kafka-compatible event streaming
• PostgreSQL — Metadata, users, projects, auth
• Next.js — Web dashboard (Tailwind CSS, TypeScript)

Development

# Run tests
make test

# Run smoke tests (stack must be up)
./scripts/smoke-test.sh

# Tidy Go modules
make tidy

Deployment

Set strong environment variables (ANALYTICS_API_AUTH_SECRET, DB credentials), enable TLS, and configure rate limits. See docker-compose.yml for all options.