fix(audio): stop Soniox writing Urdu in Devanagari — cherry-pick of #310 (bd-bfy69) - #311
Merged
Conversation
…it if it slips through (#310) Answering the question directly: Soniox is NOT returning an Urdu-English label. On a real prod session (2026-08-19) it returned 4,926 tokens tagged `hi`, 139 tagged `en`, and NOT ONE tagged `ur`. The session-level label it stored was 'en' — the minority language of its own token counts. Across the fleet the labels have been 'en', 'hindi', 'javanese' and 'sindhi' on Urdu classroom audio, and 67 sessions since 2026-08-11 came back written in Devanagari. That script reaches a coach in the FICO evidence box (HITL R62/R63/R64, with a screenshot) and a teacher in the report. It cannot be drawn: there is no Devanagari font in bot/shared/fonts/ and the render container has no system fonts, so unlike the Urdu tofu of bd-osmk0 this one cannot be fixed by naming a fallback face. Three layers, cheapest first. LAYER 1 — stop inviting it. language_hints was a hardcoded ['en','ur','es','ar','pa','ta']: six languages, four of which this deployment does not serve. Hints only BIAS Soniox's language identification, they do not restrict it, and a wide list widens the search — Urdu and Hindi are the same spoken language, so it settled on Hindi. The default now comes from LANGUAGE_OFFER (ur, en), the single source of truth for what we serve, rather than a second hardcoded list that can drift from it. Callers passing an explicit language (reading assessment) take the single-hint branch and are untouched. LAYER 2 — if it arrives anyway, re-transcribe once with a single forced `ur` hint, which leaves the identifier no room to choose Hindi. Only attempted when the caller left the language open; a caller that already pinned one would get the identical answer back. LAYER 3 — the guarantee. If it STILL arrives, transliterate to Perso-Arabic so nothing in Devanagari can ever reach a rendered surface. The language label is overwritten to 'ur' at the same time: it was part of the same wrong answer, and resolveReportLanguage reads that field to choose the report's script branch, so leaving it would send an Urdu report down the Latin arm. Layer 3 is LOSSY and says so loudly — every layer logs at level='error'. Urdu does not write short vowels, श/ष both fold to ش, ن absorbs ण, and Arabic-origin spellings are phonetic (تریکے, not طریقے). It is a legibility rescue, not a transliterator with a lexicon: a coach sees words instead of boxes. The positional rules matter and are tested — geminates collapse (بچوں not بچچوں), aspirate geminates too (اچھا not اچچھا), ے is word-final while ی is medial (میں not مےں), and a medial independent vowel takes a hamza carrier (بتائیے not بتاااے). Tests: 38 new, including a mutation pass that breaks each guard and confirms the matcher goes red. Comments are stripped before any source assertion (language-protocol §7.1). Coaching suite: 12 failing suites before and after, zero new; +38 passing. Refs: bd-bfy69 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…in's ratchet was red before this PR Unrelated to the report fix; found because it blocked the cherry-pick's CI. The logger ratchet was already failing on plain origin/main (391 live vs 389 allowlisted, verified by running it on the untouched branch tip) — the photo-gate sweep was added to main without re-baselining. develop is green because its allowlist carries these two entries. Fixed rather than grandfathered: both are genuine error paths an on-caller would want to see, and the gate's own header says a NEW violation should get the level, not the allowlist. Live count drops 391 -> 389 and matches the existing allowlist, so no re-baseline is needed and the backlog does not grow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #310, merged to
developand live on NIETE staging as7fc2cfa(bot, sqs-worker, sqs-worker-video, portal — all SUCCESS).The finding, restated
Soniox is not returning an Urdu-English label. On a real prod session (2026-08-19):
hienur…and it stored the session-level label as
en— the minority language of its own token counts. Fleet-wide those labels have beenen,hindi,javanese,sindhi. 67 sessions since 2026-08-11 came back in Devanagari.hiwas never in the hints: hints only bias the identifier, they do not restrict it.Devanagari cannot be drawn — there is no Devanagari font in
bot/shared/fonts/. Unlike #307 this cannot be fixed with a fallback face.Three layers
language_hintsnow comes fromLANGUAGE_OFFER(ur,en) instead of a hardcoded['en','ur','es','ar','pa','ta']. Callers passing an explicit language (reading assessment) are untouched.urhint.ursoresolveReportLanguagecannot send the report down the Latin arm.Every layer logs at
level='error'. Layer 3 is lossy and says so — a legibility rescue, not a transliterator with a lexicon.Two commits
7b2fdd3— the pick. Clean, no conflicts.cf4ca83— the same unrelated ratchet fix that was open as fix(report): Urdu rendered as tofu on prod whenever the STT mislabelled the language — cherry-pick of #307 #309, folded in here so prod takes one deploy rather than two. ThelogToFile severity ratchetwas already red on plainorigin/main(391 live vs 389 allowlisted); the photo-gate sweep had been added without re-baselining. Both callsites are genuine error paths and now passlevel='error', which is what the gate's own header asks for, rather than growing the grandfathered list. fix(report): Urdu rendered as tofu on prod whenever the STT mislabelled the language — cherry-pick of #307 #309 should be closed as superseded.Tests on this main-based tree
devanagari-guard24 ·devanagari-transcribe-recovery14 ·hero-report-urdu-font-fallback20 ·hero-report-language-resolution12 ·logger-level17 — all green, ratchet included.Closes: bd-bfy69
🤖 Generated with Claude Code