Skip to content

feat: dual-provider STT routing for IMBE + audio fallback (#7)#51

Open
LumenPrima wants to merge 13 commits into
masterfrom
feat/dual-provider-stt
Open

feat: dual-provider STT routing for IMBE + audio fallback (#7)#51
LumenPrima wants to merge 13 commits into
masterfrom
feat/dual-provider-stt

Conversation

@LumenPrima

Copy link
Copy Markdown
Member

Summary

  • Adds STT_FALLBACK_PROVIDER so mixed P25 + analog systems can use IMBE for .dvcf and an audio STT (whisper/elevenlabs/deepinfra) for everything else
  • Two worker pools with path-based routing; backfill and re-transcribe use the same rules
  • Documents config in sample.env, README, and AGENTS; unit tests cover routing edge cases

Closes #7

Config example

STT_PROVIDER=imbe
STT_FALLBACK_PROVIDER=whisper
IMBE_ASR_URL=http://localhost:8090
WHISPER_URL=http://localhost:8000/v1/audio/transcriptions

Without a fallback, IMBE-only behavior is unchanged (audio enqueue sites no-op; only handleDvcf enqueues).

Test plan

  • go test ./internal/ingest/ ./internal/transcribe/
  • go vet ./...
  • bash build.sh
  • Real-life smoke: enable dual-provider on a mixed system; confirm P25 calls get IMBE transcripts and analog calls get fallback transcripts
  • Backfill a window of mixed calls and confirm both providers appear in transcription records / queue stats

LumenPrima and others added 13 commits May 10, 2026 00:43
- storage: LocalStore tests (save/open round-trip, nested dirs, atomic writes,
  path traversal security, Exists, LocalPath, URL, Type, Dir, NotFound)
- trconfig: LoadConfig, LoadVolumeMap, Discover tests
- 2 new test files, 0 regressions across full suite
Bubble Scatter, Calendar Heatmap, Daily Overview (treemap), Emergency Log,
Recorder Gauges, TG Sunburst, Traffic & Patterns, Unit Tracker — all
registered via card-title/description/order meta tags.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
migrating-auth.md covers the v0.9.8 auth simplification (AUTH_ENABLED
deprecated, three-mode auto-detection). glossary-research.md captures
PostgreSQL-based phonetic/fuzzy lookup strategy for ASR post-correction.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…ards

All GET /api/v1/{systems,calls,recorders} responses are wrapped objects
({systems:[],total} etc.), not plain arrays — pages were iterating the
object directly and displaying no data.

Bugs fixed per page:
- bubble-scatter: unwrap systems/calls; tg_alpha_tag (was tgid_alpha_tag); tg_tag (was tag)
- calendar-heatmap: unwrap systems/calls
- daily-overview: unwrap systems/calls; fix unconditional j-- causing infinite loop;
  d.depth===2 for TG leaves (was d.height===2 which selected root); tg_alpha_tag; tg_tag
- tg-sunburst: unwrap systems/calls; tg_alpha_tag (was tgid_alpha_tag)
- recorder-gauges: unwrap recorders and systems
- emergency-log: fix JS syntax error (bare var(--accent) as expression); system name
  field (name not system_name); wire up trend chart on init
- traffic-patterns: system name field; call-heatmap uses days param (not hours, max 90)
- unit-tracker: unit_alpha_tag (was alpha_tag); var declaration in strict mode

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Add POST /debug-report (was implemented but missing from spec)
- Add DELETE /users/all with danger notice (bulk delete, admin only)
- Fix /admin/transcribe-backfill path param: {job_id} -> {id} to match code
- go test -race passes clean (zero DATA RACE)
- golangci-lint v1.64.8 with govet/staticcheck/errcheck/unused
- gates new code only via --new-from-rev HEAD~1
- GET /api/v1/recent-events: UNION of calls + unit_events with filters
- GET /api/v1/dashboard/summary: aggregated health/stats/active/top-TG
- Audio Range support (200/206/416) via http.ServeContent
- OpenAPI specs for all three endpoints
- Tests for handlers and audio status codes
…nto UI

- Call history: transcript search wired to server-side /transcriptions/search
- Call history: draggable seek handle with buffered range visualization
- Events page: pre-populate from /recent-events before SSE connects
- Signal Flow: surface query 403 errors instead of silent degradation
- POST /api/v1/query: explicit QueryAdminOnly middleware with clear 403
…and Playwright smoke tests

- docs/quickstart.md: Docker Compose → ingest → UI in 5 minutes
- docs/tr-timing-behavior.md: call ID shifts, unit_event:end lag, mitigations
- docs/release-notes.md: v0.10.0 release notes with features/fixes/upgrade notes
- tests/: Playwright smoke suite (8 tests) against deployed instance
W1: Refresh token rotation — JTI-based rotation with reuse detection
  - New migration adds refresh_token_jti column to users table
  - Refresh handler validates JTI, rejects reused tokens with 401
  - Logout clears stored JTI before expiring cookie

W2: rewriteInstanceID() — replace fragile byte-level JSON with
  json.Unmarshal/json.Marshal round-trip, preventing false matches
  on keys like my_instance_id

W3: AGENTS.md — fix fuzzy match tolerance documentation (±5s → ±10s)

W4: AuthRateLimiter — fix goroutine leak via time.NewTicker with
  ctx.Done() select, ShutdownCtx propagated from main

W5: Middleware chain — swap RateLimiter/Recoverer order so panics
  in rate limiter are caught by Recoverer
When STT_PROVIDER=imbe, analog/conventional calls previously got no
transcription. Add STT_FALLBACK_PROVIDER so .dvcf jobs go to IMBE and
non-DVCF audio jobs route to a second worker pool (whisper/elevenlabs/
deepinfra). Backfill and re-transcribe use the same path-based routing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dual-provider transcription routing (IMBE + fallback)

1 participant