Skip to content

Proposal: training-corpus de-identification mode with erasure-by-key (crypto-shredding) #66

Description

@mazzasaverio

Context

Machine unlearning does not work at production scale: retraining costs millions, approximate unlearning is immature research (arXiv 2508.12220, 2503.01630), and regulators increasingly treat model weights as personal data. Meanwhile every enterprise is starting to fine-tune on its own corpora, and GDPR Art. 17 erasure requests collide head-on with "the data is baked into the weights".

There is one practical way out, and pseudonymize already owns the primitive for it: de-identify the corpus before training, with deterministic aliases, and make erasure a key-management operation instead of an ML operation.

  • Deterministic HMAC aliases preserve referential integrity across the whole corpus: <PERSON_1> is the same token in every document, so the fine-tuned model still learns coherent entity patterns; utility survives where naive redaction destroys it.
  • Erasure becomes crypto-shredding: destroy the key (or the namespace's key material) and every alias in the trained model becomes an unlinkable token. With per-partition namespaces (one per customer, patient, or data subject grouping the caller already has), single-subject erasure is dropping one derived key.

Proposal

  1. A corpus/batch API that guarantees alias consistency across many documents processed over time (a persistent AliasContext contract: serialization rules, concurrency notes, what is and is not sensitive in it).
  2. A namespace-derivation helper for per-partition keys (e.g. derive_namespace(master_key, partition_id)), so callers get subject-scoped erasure without inventing their own key scheme.
  3. A documented workflow page: pseudonymize → fine-tune → serve; what erasure-by-key does and does not achieve (it removes linkability, it is not anonymization; existing non-goals language applies); train/validation split hygiene.
  4. Nothing new in the trusted core beyond the helper: this is packaging existing primitives (deterministic mode, namespaces, batch processing) at the training-data control point.

Why this matters at five years

Inference-time redaction is where the market is today; the training-data pipeline is where it moves, because that is where the un-erasable liability is created. EDPB's 2026 anonymisation draft and the models-as-personal-data literature make this collision explicit. A local, deterministic, auditable de-identification step in front of fine-tuning is the one control that makes erasure tractable, and no current OSS tool packages it as a workflow.

🤖 Generated with Claude Code

https://claude.ai/code/session_012JiX3zWeEC28kmy5KmAXvf

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions