You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b525e27
Browse filesBrowse the repository at this point in the historyBrowse files
fix(pii): cap each batched NER pass at one window of characters (#10099)
A batch of many texts each under NER_WINDOW_CHARS still went through one
nlp.pipe pass, and that pass costs memory for every character in it, like one
long text. Pack the short texts into batches of at most NER_WINDOW_CHARS in
total and take the NER slot per batch.
0 commit comments