feat: implement end-to-end AI meeting insights pipeline#3
Merged
Conversation
This commit introduces the complete infrastructure and UI for real-time transcription and AI-powered meeting analysis. Key Changes: Go Backend (minutely-api): - Implemented asynchronous worker pool for AI processing jobs. - Added WebSocket Hub for real-time segment broadcasting and DB persistence. - Fixed 422 errors by aligning AIEvent JSON tags with Python snake_case. - Added safety check to skip AI processing for sessions with 0 segments. - Moved API to port 8081 to avoid conflict with webpack-dev-server. - Added HTTP handlers for transcript retrieval and AI insight generation. - Implemented repositories for meeting metadata, transcripts, and AI outputs. AI Service (ai-service): - Deployed BART-based summarization service using Modal. - Improved Pydantic schema resilience with extra="ignore". - Added fallback logic to join segments into full text if aggregation is missing. Frontend (React/Minutely): - Integrated Deepgram-based live transcription in middleware. - Added 'Meeting Intelligence' sidebar with Live and AI Insights views. - Implemented ToggleSidebarButton and custom UI components. - Configured webpack proxy to route API and WebSocket traffic to port 8081. Infrastructure: - Added consolidated SQL migrations for Supabase schema alignment. - Updated .gitignore to exclude local scratch/debug directories.
There was a problem hiding this comment.
Pull request overview
Implements an end-to-end “AI meeting insights” pipeline spanning the React UI, a Go backend (HTTP + WebSocket + workers + Supabase persistence), and a Python/Modal AI service for summarization/topics/action items.
Changes:
- Added React transcription feature (middleware + Redux + sidebar UI) and wired it into the conference view.
- Added Go backend support for live transcription sessions (WS hub), recording upload → async processing, and AI insights retrieval.
- Added/updated Supabase schema migrations and introduced an AI service deployable via Modal.
Reviewed changes
Copilot reviewed 39 out of 41 changed files in this pull request and generated 24 comments.
Show a summary per file
| File | Description |
|---|---|
| webpack.config.js | Adds dev-server proxy routing /api/v1 (incl. WS) to the Go API on :8081. |
| react/features/transcription/reducer.ts | Introduces Redux state for transcription segments and sidebar open/close state. |
| react/features/transcription/middleware.ts | Implements Deepgram live capture + forwarding segments to Go WS hub + session start/stop. |
| react/features/transcription/index.ts | Registers transcription feature exports and side-effect imports (reducer/middleware). |
| react/features/transcription/components/index.ts | Exports transcription UI components. |
| react/features/transcription/components/UploadRecordingModal.tsx | Adds UI for uploading recordings to the backend for async transcription. |
| react/features/transcription/components/TranscriptPanel.tsx | Adds “Meeting Intelligence” sidebar with Live + AI Insights views. |
| react/features/transcription/components/ToggleSidebarButton.tsx | Adds a button to toggle the Meeting Intelligence sidebar. |
| react/features/transcription/actions.ts | Adds Redux actions for start/stop/status/segment receipt and toggling UI. |
| react/features/transcription/actionTypes.ts | Adds action type constants for transcription feature. |
| react/features/conference/components/web/Conference.tsx | Mounts the transcript panel and toggle button in the conference UI. |
| react/features/app/reducers.any.ts | Registers the new transcription reducer. |
| react/features/app/middlewares.web.ts | Registers the new transcription middleware. |
| minutely-api/supabase/migrations/000002_consolidated_schema.sql | Adds a consolidated schema migration for meetings/transcripts/jobs/AI outputs with RLS. |
| minutely-api/internal/workers/pool.go | Adds a fixed-size worker pool polling pending jobs. |
| minutely-api/internal/workers/file_processor.go | Adds upload transcription processing (storage download → ffmpeg → Modal → persist → AI hook). |
| minutely-api/internal/transport/ws/hub.go | Adds a WS hub to broadcast segments and persist final segments to DB. |
| minutely-api/internal/transport/ws/client.go | Adds WS client read/write pumps and connection upgrade handling. |
| minutely-api/internal/transport/http/transcription_handler.go | Adds upload endpoints, job status/progress (SSE), and transcript retrieval. |
| minutely-api/internal/transport/http/live_transcription_handler.go | Adds live transcription start/end endpoints and meeting WS endpoint handler. |
| minutely-api/internal/transport/http/ai_handler.go | Adds endpoint to fetch AI insights for a meeting. |
| minutely-api/internal/core/services/transcription_service.go | Implements start/end live session logic and AI job enqueueing. |
| minutely-api/internal/core/domain/transcript.go | Defines transcript domain models and repository/service interfaces. |
| minutely-api/internal/core/domain/storage.go | Defines storage service interface for upload/download/signed URLs. |
| minutely-api/internal/core/domain/job.go | Defines job/media domain models and repository/service interfaces. |
| minutely-api/internal/core/domain/ai_output.go | Defines AI output domain models and AI processor/repository interfaces. |
| minutely-api/internal/adapters/storage/supabase_storage.go | Implements storage adapter on top of storage-go. |
| minutely-api/internal/adapters/pythonai/ai_processor.go | Implements AI processor calling Python service and persisting AI outputs. |
| minutely-api/internal/adapters/postgres/transcript_repo.go | Implements transcript persistence via Supabase/PostgREST. |
| minutely-api/internal/adapters/postgres/job_repo.go | Implements job/media persistence via Supabase/PostgREST. |
| minutely-api/internal/adapters/postgres/ai_output_repo.go | Implements AI output persistence via Supabase/PostgREST. |
| minutely-api/internal/adapters/modal/client.go | Adds client for Modal transcription endpoint (multipart upload). |
| minutely-api/internal/adapters/deepgram/client.go | Adds Deepgram client (currently stubbed key generation). |
| minutely-api/internal/adapters/audio/ffmpeg.go | Adds ffmpeg-based audio extraction to 16kHz mono WAV. |
| minutely-api/go.sum | Adds/updates Go dependency checksums for new backend functionality. |
| minutely-api/go.mod | Adds/updates Go module dependencies and Go version directive. |
| minutely-api/cmd/api/main.go | Wires repositories/adapters/services/routes, starts worker pool and WS hub, moves API to :8081. |
| ai-service/requirements.txt | Adds Python AI service dependencies. |
| ai-service/modal_app.py | Adds Modal-deployed FastAPI service with cached model loading and AI pipeline. |
| ai-service/app.py | Adds local FastAPI service implementation of the AI pipeline. |
| .gitignore | Ignores scratch/debug directories and Python bytecode caches. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| }) | ||
| }) | ||
|
|
||
| r.Route("/jobs", func(r chi.Router) { |
Comment on lines
+84
to
+88
| function stopTranscriptionPipeline(store: any) { | ||
| if (deepgramWs) { | ||
| deepgramWs.close(); | ||
| deepgramWs = null; | ||
| } |
Comment on lines
+161
to
165
| // --- Start Server --- | ||
| log.Println("Starting Minutely API on :8081") | ||
| if err := http.ListenAndServe(":8081", r); err != nil { | ||
| log.Fatalf("Server failed to start: %v", err) | ||
| } |
Comment on lines
+31
to
+35
| // TEMPORARY PROTOTYPE FIX: | ||
| // The provided DEEPGRAM_KEY lacks "Administrator" permissions to generate temporary scoped keys (returns 403 Forbidden). | ||
| // For testing purposes, we will just return the master key to the frontend. | ||
| // In production, the DEEPGRAM_KEY must be an Administrator key to generate temporary scoped tokens. | ||
| return c.apiKey, nil |
Comment on lines
+20
to
+24
| var upgrader = websocket.Upgrader{ | ||
| ReadBufferSize: 1024, | ||
| WriteBufferSize: 1024, | ||
| CheckOrigin: func(r *http.Request) bool { | ||
| return true // Allow all origins for now; restrict in production |
Comment on lines
+68
to
+69
| h.mu.Unlock() | ||
| log.Printf("Client registered to meeting %s. Total clients in meeting: %d", client.MeetingID, len(h.clients[client.MeetingID])) |
Comment on lines
+46
to
+47
| _, _ = h.meetingService.GetMeeting(r.Context(), meetingID) | ||
| err = h.meetingService.CreateMeeting(r.Context(), &domain.Meeting{ |
| if err == nil { | ||
| transcript.Status = domain.TranscriptStatusCompleted | ||
| transcript.CompletedAt = &now | ||
| _ = s.repo.Update(ctx, transcript) |
Comment on lines
+63
to
+66
| func (h *TranscriptionHandler) UploadRecording(w http.ResponseWriter, r *http.Request) { | ||
| _ = r.Context().Value(middleware.UserIDKey).(uuid.UUID) | ||
|
|
||
| meetingID := h.parseOrGenerateMeetingID(r) |
Comment on lines
+38
to
+42
| client := &http.Client{} | ||
| resp, err := client.Do(req) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to call python AI service: %w", err) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit introduces the complete infrastructure and UI for real-time transcription and AI-powered meeting analysis.
Key Changes:
Go Backend (minutely-api):
AI Service (ai-service):
Frontend (React/Minutely):
Infrastructure: