fix(audio): stop Soniox writing Urdu in Devanagari, and never render it if it slips through (bd-bfy69) - #310
Merged
Merged
Conversation
…it if it slips through Answering the question directly: Soniox is NOT returning an Urdu-English label. On a real prod session (2026-08-19) it returned 4,926 tokens tagged `hi`, 139 tagged `en`, and NOT ONE tagged `ur`. The session-level label it stored was 'en' — the minority language of its own token counts. Across the fleet the labels have been 'en', 'hindi', 'javanese' and 'sindhi' on Urdu classroom audio, and 67 sessions since 2026-08-11 came back written in Devanagari. That script reaches a coach in the FICO evidence box (HITL R62/R63/R64, with a screenshot) and a teacher in the report. It cannot be drawn: there is no Devanagari font in bot/shared/fonts/ and the render container has no system fonts, so unlike the Urdu tofu of bd-osmk0 this one cannot be fixed by naming a fallback face. Three layers, cheapest first. LAYER 1 — stop inviting it. language_hints was a hardcoded ['en','ur','es','ar','pa','ta']: six languages, four of which this deployment does not serve. Hints only BIAS Soniox's language identification, they do not restrict it, and a wide list widens the search — Urdu and Hindi are the same spoken language, so it settled on Hindi. The default now comes from LANGUAGE_OFFER (ur, en), the single source of truth for what we serve, rather than a second hardcoded list that can drift from it. Callers passing an explicit language (reading assessment) take the single-hint branch and are untouched. LAYER 2 — if it arrives anyway, re-transcribe once with a single forced `ur` hint, which leaves the identifier no room to choose Hindi. Only attempted when the caller left the language open; a caller that already pinned one would get the identical answer back. LAYER 3 — the guarantee. If it STILL arrives, transliterate to Perso-Arabic so nothing in Devanagari can ever reach a rendered surface. The language label is overwritten to 'ur' at the same time: it was part of the same wrong answer, and resolveReportLanguage reads that field to choose the report's script branch, so leaving it would send an Urdu report down the Latin arm. Layer 3 is LOSSY and says so loudly — every layer logs at level='error'. Urdu does not write short vowels, श/ष both fold to ش, ن absorbs ण, and Arabic-origin spellings are phonetic (تریکے, not طریقے). It is a legibility rescue, not a transliterator with a lexicon: a coach sees words instead of boxes. The positional rules matter and are tested — geminates collapse (بچوں not بچچوں), aspirate geminates too (اچھا not اچچھا), ے is word-final while ی is medial (میں not مےں), and a medial independent vowel takes a hamza carrier (بتائیے not بتاااے). Tests: 38 new, including a mutation pass that breaks each guard and confirms the matcher goes red. Comments are stripped before any source assertion (language-protocol §7.1). Coaching suite: 12 failing suites before and after, zero new; +38 passing. Refs: bd-bfy69 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hyasin270
added a commit
that referenced
this pull request
Aug 19, 2026
(bd-bfy69) (#311) * fix(audio): stop Soniox writing Urdu in Devanagari, and never render it if it slips through (#310) Answering the question directly: Soniox is NOT returning an Urdu-English label. On a real prod session (2026-08-19) it returned 4,926 tokens tagged `hi`, 139 tagged `en`, and NOT ONE tagged `ur`. The session-level label it stored was 'en' — the minority language of its own token counts. Across the fleet the labels have been 'en', 'hindi', 'javanese' and 'sindhi' on Urdu classroom audio, and 67 sessions since 2026-08-11 came back written in Devanagari. That script reaches a coach in the FICO evidence box (HITL R62/R63/R64, with a screenshot) and a teacher in the report. It cannot be drawn: there is no Devanagari font in bot/shared/fonts/ and the render container has no system fonts, so unlike the Urdu tofu of bd-osmk0 this one cannot be fixed by naming a fallback face. Three layers, cheapest first. LAYER 1 — stop inviting it. language_hints was a hardcoded ['en','ur','es','ar','pa','ta']: six languages, four of which this deployment does not serve. Hints only BIAS Soniox's language identification, they do not restrict it, and a wide list widens the search — Urdu and Hindi are the same spoken language, so it settled on Hindi. The default now comes from LANGUAGE_OFFER (ur, en), the single source of truth for what we serve, rather than a second hardcoded list that can drift from it. Callers passing an explicit language (reading assessment) take the single-hint branch and are untouched. LAYER 2 — if it arrives anyway, re-transcribe once with a single forced `ur` hint, which leaves the identifier no room to choose Hindi. Only attempted when the caller left the language open; a caller that already pinned one would get the identical answer back. LAYER 3 — the guarantee. If it STILL arrives, transliterate to Perso-Arabic so nothing in Devanagari can ever reach a rendered surface. The language label is overwritten to 'ur' at the same time: it was part of the same wrong answer, and resolveReportLanguage reads that field to choose the report's script branch, so leaving it would send an Urdu report down the Latin arm. Layer 3 is LOSSY and says so loudly — every layer logs at level='error'. Urdu does not write short vowels, श/ष both fold to ش, ن absorbs ण, and Arabic-origin spellings are phonetic (تریکے, not طریقے). It is a legibility rescue, not a transliterator with a lexicon: a coach sees words instead of boxes. The positional rules matter and are tested — geminates collapse (بچوں not بچچوں), aspirate geminates too (اچھا not اچچھا), ے is word-final while ی is medial (میں not مےں), and a medial independent vowel takes a hamza carrier (بتائیے not بتاااے). Tests: 38 new, including a mutation pass that breaks each guard and confirms the matcher goes red. Comments are stripped before any source assertion (language-protocol §7.1). Coaching suite: 12 failing suites before and after, zero new; +38 passing. Refs: bd-bfy69 Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * chore(logger): the two photo-gate ❌ calls now pass level='error' — main's ratchet was red before this PR Unrelated to the report fix; found because it blocked the cherry-pick's CI. The logger ratchet was already failing on plain origin/main (391 live vs 389 allowlisted, verified by running it on the untouched branch tip) — the photo-gate sweep was added to main without re-baselining. develop is green because its allowlist carries these two entries. Fixed rather than grandfathered: both are genuine error paths an on-caller would want to see, and the gate's own header says a NEW violation should get the level, not the allowlist. Live count drops 391 -> 389 and matches the existing allowlist, so no re-baseline is needed and the backlog does not grow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Answering the question first
Soniox is not returning an Urdu-English label. On a real prod session (2026-08-19) it returned:
hienur…and stored the session-level label as
en— the minority language of its own token counts. Across the fleet those labels have beenen,hindi,javaneseandsindhion Urdu classroom audio. 67 sessions since 2026-08-11 came back written in Devanagari.Note
hiwas never in the hints. Hints only bias the language identifier; they do not restrict it.Why it must never render
There is no Devanagari font in
bot/shared/fonts/, and the render container has no system fonts. Unlike the Urdu tofu in #307, this cannot be fixed by naming a fallback face — we do not ship the face. It reaches a coach in the FICO evidence box (R62/R63/R64, with a screenshot) and a teacher in the report.Three layers, cheapest first
language_hintswas a hardcoded['en','ur','es','ar','pa','ta']— six languages, four of which this deployment does not serve. The default now comes fromLANGUAGE_OFFER(ur,en), the single source of truth, rather than a second hardcoded list that can drift from it. Callers passing an explicit language (reading assessment) take the single-hint branch and are untouched.urhint, leaving the identifier no room to choose Hindi.urat the same time; it was part of the same wrong answer, andresolveReportLanguagereads that field to choose the report's script branch.Every layer logs at
level='error'.Layer 3 is lossy, and says so
It is a legibility rescue, not a transliterator with a lexicon — a coach sees words instead of boxes. Urdu does not write short vowels; श/ष both fold to ش; ण and न both fold to ن; Arabic-origin spellings come out phonetic (
تریکے, notطریقے). The positional rules are where the quality actually lives, and each is tested:Tests
38 new. Includes a mutation pass that breaks each guard in memory and confirms the matcher goes red — no vacuous assertions. Comments are stripped before any source assertion (language-protocol §7.1).
Coaching suite: 12 failing suites before and after, zero new; +38 passing.
circular-depsandunresolved-requiresfail identically on untoucheddevelop.Refs: bd-bfy69
🤖 Generated with Claude Code