Skip to content

Proposal: logging and OpenTelemetry integration (redact in-process, before export) #67

Description

@mazzasaverio

Context

Logs and telemetry are the highest-volume real-world PII leak channel, and the 2026 guidance for it is uniformly weak: every OpenTelemetry article recommends regex processors in the Collector, which means (a) crude detection and (b) the data has already left the process unredacted. Datadog-style sensitive-data scanners are server-side and paid, and scan after ingestion.

The vision's principle 5 ("never make observability another leak") is currently an internal invariant. This proposal turns it into the product's cheapest distribution wedge: every Python application logs, and almost none of them sanitize.

Proposal

Two thin integrations, each an optional extra, both applying the existing engine:

  1. pseudonymize.logging.RedactingHandler / RedactingFilter for the standard library: wraps any handler, pseudonymizes or redacts record messages and formatted args before emit. Zero dependencies, works with structlog/loguru via the stdlib bridge. Fail-closed option: on engine error, drop or replace the record rather than emit raw.
  2. An OTel processor (pseudonymize[otel]): a LogRecordProcessor and SpanProcessor that sanitize attributes and events in-process, before the exporter, with per-attribute-path policies (the JSON-path policy machinery maps directly onto attribute names).

Notes:

  • In-process beats Collector regexes on both detection quality and trust boundary (nothing unredacted ever crosses the wire).
  • Performance matters here more than anywhere: the ASCII fast path from fix: sanitize ONNX backend errors and restore the green formatting gate #57 helps; a "structured detectors only" preset (no ML) should be the documented default for hot logging paths.
  • Deterministic mode keeps logs debuggable: the same user is <EMAIL_1> across the whole trace, which regex masking destroys.

Why this wins

It meets developers where they already have the problem, requires no architectural buy-in (one handler line), and every install seeds familiarity with the rest of the boundary (files, LLM round-trip). Cross-border/data-residency pressure makes "sanitize before export" the pattern regulators point at.

🤖 Generated with Claude Code

https://claude.ai/code/session_012JiX3zWeEC28kmy5KmAXvf

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions