Skip to content

Fix same-line rank/PIB corruption; consistent surname stems; ending counts toward the half-limit (v3.0.14–3.0.16) - #56

Merged
click0 merged 3 commits into
mainfrom
claude/refactor-data-masking-lIcWN
Sep 8, 2026
Merged

click0 merged 3 commits into
mainfrom
claude/refactor-data-masking-lIcWN

Conversation

@click0

@click0 click0 commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Summary

Three fixes driven by a real counterexample:

was:  довідках №273 рядового МАЗУРЕНКА
got:  довідках №857 старшого рядового МАЗИДЕНКА

v3.0.14 — engine: several ranks / PIBs on one line no longer corrupt each other

Masks were substituted into the partially masked line with str.replace(original, mask, 1). When the mask of one rank contained the form of another rank on the same line (рядовий → старший солдат, солдат → рядовий), the second substitution hit the freshly inserted mask: рядового МАЗУРЕНКА та солдата КОВАЛЕНКАстаршого рядового МАЗИДЕНКА та солдата КОВИЛЕНКА (non-existent rank, second rank left unmasked, unmask restored the wrong rank). Replacements are now numbered placeholders in the working copy of the line, substituted once at the end. Same fix for full-name replacements. New tests: tests/test_same_line_replacement.py.

v3.0.15 — surname: one person, one synthetic stem

The synthetic stem was seeded from the surface form, so МАЗУРЕНКА / Мазуренка / Мазуренко got three unrelated masks. The seed is now the lower-cased stem; all letter-case and grammatical-case forms share one stem, and ending + case are applied on top. Unmask is mapping-driven and unaffected.

v3.0.16 — surname prefix: the preserved ending counts toward "at most half"

Only the prefix was capped at half the word, so with a long ending most of the original stayed visible (Мазуренка → Мазиденка: 7 of 9 letters). Prefix + preserved ending are now limited to half of the base form: Коваль → 3, Ґудзь → 2, Іванов → 1, Бондаренко → 1, Петренко → 0. Computed from the base form so all cases of one surname keep the same prefix. Docs, config template and tests updated.

Test plan

  • pytest tests/ — 810 passed, 1 skipped
  • mypy datamasking/ — 0 errors
  • flake8 --select=E9,F63,F7,F82
  • Manual: the counterexample line masks to старшого солдата … та рядового … and unmasks to the original
  • CI green on the full matrix

🤖 Generated with Claude Code

https://claude.ai/code/session_01XT6iUWaQgahXDB9TWX9Bq7


Generated by Claude Code

…her (v3.0.14)

Masks were substituted into the partially masked line with
str.replace(original, mask, 1); when the mask of one rank contained the
form of another rank on the same line, the second substitution hit the
freshly inserted mask ("рядового ... солдата" -> "старшого рядового",
second rank left unmasked, unmask restored the wrong rank). Replacements
are now numbered placeholders in the working copy of the line and are
substituted once at the end.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XT6iUWaQgahXDB9TWX9Bq7
Generated-With: Claude Code 2.1.42
….15)

МАЗУРЕНКА / Мазуренка / Мазуренко (one person in upper case, title
case and different grammatical cases) got three unrelated masks because
the seed was taken from the surface form. All forms now share one
synthetic stem; ending and letter case are applied on top. Unmask is
mapping-driven and unaffected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XT6iUWaQgahXDB9TWX9Bq7
Generated-With: Claude Code 2.1.42
…rname" limit (v3.0.16)

Only the prefix was capped at half of the word, so with a long ending
most of the original stayed visible (Мазуренка -> Мазиденка kept 7 of 9
letters). Prefix + preserved ending are now limited to half of the base
form of the surname; the budget is computed from the base form so all
grammatical cases keep the same prefix.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XT6iUWaQgahXDB9TWX9Bq7
Generated-With: Claude Code 2.1.42
@click0
click0 merged commit 1115fc5 into main Sep 8, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants