Speak Up

IELTS speaking practice app with a NestJS API, Next.js frontend, and fully local AI feedback (Whisper + Ollama). Audio, transcripts, and scores stay on your machine — no cloud AI APIs.

Stack

• Web: Next.js 15, React 19, Tailwind CSS 4
• API: NestJS 11, Prisma, PostgreSQL, BullMQ, Redis, MinIO, Socket.IO
• AI: local Faster Whisper (transcription) + Ollama (IELTS-style scoring)

Prerequisites

Tool	Notes
Node.js 20+	Required
Yarn 4	Enabled via Corepack (corepack enable) — the repo pins yarn@4.16.0
Docker	Docker Desktop or OrbStack must be running
Ollama	Install from ollama.com, then pull a model

Optional but recommended: ~16GB+ RAM if you run Whisper + qwen2.5:7b together.

Ports

Service	URL / port
Web	http://localhost:3001
API	http://localhost:4000
API docs (Swagger)	http://localhost:4000/docs
AI health	http://localhost:4000/api/v1/health/ai
PostgreSQL	localhost:5433 (mapped from container 5432)
Redis	localhost:6379
MinIO API / console	9000 / 9001
Whisper	localhost:8001
Ollama	localhost:11434 (host process, not Docker)

Quick start

Run all commands from the repository root.

1. Enable Yarn and install dependencies

corepack enable
yarn install

2. Create env files

cp .env.example .env
cp .env.example apps/api/.env
cp .env.example apps/web/.env.local

The API reads apps/api/.env. Keep DATABASE_URL, Redis, MinIO, and AI URLs pointing at localhost as in .env.example.

Default AI flags in .env.example:

• WHISPER_MOCK=true — uses a sample transcript (no Whisper container required)
• Set WHISPER_MOCK=false in both .env and apps/api/.env once Whisper is running and you want real STT

Optional: OLLAMA_MOCK=true skips Ollama and returns mock scores (useful for UI-only work).

3. Start Docker services

Make sure Docker/OrbStack is running, then:

docker compose up -d

This starts Postgres, Redis, MinIO, MinIO bucket init, and Whisper.

Check status:

docker compose ps

Whisper may take a minute on first start while it pulls the image and loads the tiny model. Confirm it is up:

curl http://localhost:8001/health
# or
curl http://localhost:8001/v1/models

Opening http://localhost:8001 in a browser shows the Gradio UI. Submit only after selecting/uploading audio — an empty submit crashes that UI with a NoneType path error. Your app uses the API (/v1/audio/transcriptions), not that UI.

4. Start Ollama and pull a model

In a separate terminal:

ollama serve

In another terminal (or after the server is up):

ollama pull qwen2.5:7b

qwen2.5:7b is the default (OLLAMA_MODEL). On machines with 32–48GB RAM you can try qwen2.5:14b for stronger feedback.

5. Build shared types, sync DB, seed

yarn workspace @speak-up/shared build
yarn db:generate
yarn db:push
yarn db:seed

6. Run the apps

# Terminal A — API (port 4000)
yarn dev:api

# Terminal B — Web (port 3001)
yarn dev:web

Or run both from the root:

yarn dev

Open http://localhost:3001, register an account, then start a practice session.

Verify the setup

# Infrastructure
docker compose ps
curl -s http://localhost:4000/api/v1/health

# AI readiness (Ollama + Whisper + prosody / mocks)
curl -s http://localhost:4000/api/v1/health/ai | jq

In the Practice UI, the AI status banner should show ready when all services (or mocks) are OK.

Local AI modes

Goal	Config	What to run
Fastest first run (no STT)	WHISPER_MOCK=true (default in .env.example)	Docker without waiting on Whisper; still run Ollama for real scores
Real transcription	WHISPER_MOCK=false in .env and apps/api/.env, restart API	docker compose up -d (includes Whisper)
Real fluency / prosody	PROSODY_MOCK=false in .env and apps/api/.env, restart API	docker compose up -d --build prosody-scorer
UI only / no LLM	OLLAMA_MOCK=true	No ollama serve needed

After changing env files, restart yarn dev:api.

Inspect the database

Postgres credentials (from Compose / .env):

• Host: localhost
• Port: 5433 (not 5432)
• User / password / DB: speakup / speakup / speakup

# Ensure Postgres is up
docker compose up -d postgres

# psql inside the container
docker compose exec postgres psql -U speakup -d speakup

Useful SQL once connected:

\dt
SELECT id, email, "displayName" FROM "User" LIMIT 10;
SELECT id, status, part FROM "PracticeSession" ORDER BY "startedAt" DESC LIMIT 10;

Connection URL:

postgresql://speakup:speakup@localhost:5433/speakup

Monorepo layout

apps/web          Next.js UI
apps/api          NestJS API + Prisma schema + BullMQ worker
packages/shared   Shared TypeScript types / DTOs

Core flow

1. Register / login (JWT access + refresh tokens in httpOnly cookies)
2. Pick Part 1 / 2 / 3 and a topic (or generate a full speaking test)
3. Record audio in the browser
4. Upload to MinIO via a presigned URL
5. BullMQ worker: Whisper transcript → Ollama IELTS-style scores
6. WebSocket pushes session status updates to the UI

Note: Scoring uses the transcript text only (not the raw audio). Lexical and grammar criteria work well from text; fluency and pronunciation are inferred from the transcript and are weaker proxies than a real examiner with audio.

Useful commands

# All Docker services (foreground logs)
docker compose up

# Stop everything
docker compose down

# API + web together
yarn dev

# Rebuild shared package after type changes
yarn workspace @speak-up/shared build

Further reading

See TECHNICAL_REPORT.md for the full tech stack, database schema (tables, fields, 3NF notes), and architecture direction.