You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
compute_question_set_hash raises UnicodeEncodeError for a question with a lone-surrogate escape #757
compute_question_set_hash in dokimi/dokimi/orchestrator/pipeline.py:35 joins the sorted id:question fingerprints and encodes them with .encode("utf-8"). A questions file with a \ud800 or \udcff JSON escape loads as a lone surrogate. Repro: compute_question_set_hash([{'id': 'q', 'question': json.loads('"n\\ud800"')}]) raises UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position N: surrogates not allowed. Fix: .encode('utf-8', 'surrogatepass'), as #720 does for the other digests of parsed data; the hash of every question set that encodes today is unchanged.
Activity