Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
68 lines (57 loc) · 4.1 KB

ARCHITECTURE.md

File metadata and controls

68 lines (57 loc) · 4.1 KB
Raw

Architecture

Testify converts a regulated phone call into one structured, validated ComplianceScorecard. The design goal is meaningful use of AMD compute for the throughput-bound perception work (batch ASR), with Gemma doing the compliance reasoning — while staying runnable anywhere via pluggable providers + a mock mode.

Pipeline

POST /api/v1/audits {url, ruleset?}  ─┐
POST /api/v1/audits/upload           ─┴─►  Job (queued)  ──►  async worker (thread)
                                                              │
   1. INGEST      yt-dlp (or httpx for a direct media link) → the call recording
   2. AUDIO       ffmpeg → 16 kHz mono WAV(s). If the source is 2-channel (dual-channel
                  call recording): split ch0 = AGENT, ch1 = CUSTOMER → speaker attribution
                  with NO ML diarization. Mono → one 'unknown' track.
   3. ASR         Whisper-large-v3 (transformers) per channel  ── AMD MI300X / ROCm ──┐ heavy
                  → segments tagged with speaker, merged by start time                │ GPU work
   4. GRADE       Gemma grades EVERY rule in the rule set → pass/fail/insufficient,    │
                  each with a verbatim timestamped speaker-attributed quote (temp 0)   │
   5. VALIDATE    postprocess: clamp exposure to fails only, flag insufficient/low-conf│
                  for human review, derive overall_verdict, compute the risk rollup   ─┘
   6. PERSIST     SQLite (jobs + audits)

GET /api/v1/jobs/{id}                   poll status/progress
GET /api/v1/jobs/{id}/events            live SSE progress
GET /api/v1/audits/{id}                 the ComplianceScorecard
GET /api/v1/audits/{id}/evidence-pack   court-ready evidence pack (JSON)

Why this split

  • Perception on the GPU. ASR is the throughput-bound workload that justifies an MI300X: its 192 GB HBM3 lets you batch enormous overnight call volumes through Whisper-large-v3 at tens-of-x realtime. That throughput is what makes auditing 100% of calls (vs. a ~2% manual sample) economically viable — it is the ROI, and it's the "Use of AMD Platforms" story. The realtime_factor (audio seconds / wall-clock seconds) is reported on every scorecard.
  • Reasoning on Gemma. Grading a transcript against a rule set is a pure text reasoning + structured-output task — no images — so Gemma's text-only serving path is exactly right (no multimodal workaround needed). Pointing SYNTHESIZER=amd at a local vLLM-served Gemma on the MI300X makes it AMD-hosted Gemma (Track-3 Gemma prize).
  • Trust by construction. The model is told to quote verbatim and to return insufficient_evidence rather than guess; the server then enforces the invariants it can't trust an LLM to always hold (exposure only on fails, human-review routing, verdict derivation).

Provider matrix

Stage Providers Default (MI300X) Fallback
Transcribe local (Whisper/transformers), mock local (GPU) CPU → mock
Synthesize fireworks (Gemma via Fireworks/Google), amd (vLLM Gemma), mock fireworks or amd mock

Every real provider is lazily imported and wrapped so a missing dependency, key, or GPU degrades gracefully instead of crashing. MOCK_MODE=true forces the whole chain offline (bundled fixtures) — used for the zero-setup demo and CI.

Key modules

  • app/services/pipeline.py — orchestrator + graceful degradation.
  • app/services/audio.py — ffprobe channel detection + dual-channel agent/customer split.
  • app/services/{transcribe,synthesize}/ — provider packages, each with a build_*() factory.
  • app/services/postprocess.py — normalize() (verdict/exposure/review invariants) + compute_risk().
  • app/models/audit.py — ComplianceScorecardContent (LLM schema) + ComplianceScorecard (envelope) + to_evidence_pack() / to_evidence_html(). Note the Verdict.pass_ = "pass" trick (pass is a Python keyword; it still serializes to the JSON string "pass").
  • app/worker.py — async job manager + SSE fan-out.
  • app/store/store.py — SQLite persistence.