[Integration]: Sequencer middleware.pii_mask step op #58

Activity

1. rosspeili

   added

   enhancementNew feature or improvement request.New feature or improvement request.

   securitySecurity, trust model, or policy bypass concerns.Security, trust model, or policy bypass concerns.

   sequencerPrescriptive step pipelines, gates, step telemetry.Prescriptive step pipelines, gates, step telemetry.

   on Aug 26, 2026
2. rosspeili

   added the

   Feature

   issue type on Aug 26, 2026
3. rosspeili

   added

   membraneIngress context and egress guarded tool calls.Ingress context and egress guarded tool calls.

   testingpytest, black, flake8, or test coverage.pytest, black, flake8, or test coverage.

   on Aug 26, 2026
4. rosspeili commented on Oct 11, 2026

   

   rosspeili

   on Oct 11, 2026

   ContributorAuthor

   More actions

   Context & Architectural Motivation

   Issue #58 originally stated:

   Implement middleware.pii_mask sequencer step, basically redact email/phone/key-like patterns at step boundaries, and emit middleware.applied on spine. Documented in ops README, not yet implemented.

   While the intent is clear, turning this into a production-grade feature requires bridging AURA's core design tenets with enterprise PII standards (NIST SP 800-122, GDPR Art. 32, HIPAA Safe Harbor).

   Currently, AURA has:

   1. Key-name payload redaction (aura.core.payload_redaction): Redacts known credential key names (token, api_key, password) on spine emit.
   2. Middleware schema stub (spec/middleware-policy.schema.json & aura.sequencer.middleware): Declares scope: per_step and order: [{op: "...", config: ...}], but MiddlewareStack._invoke is an empty pass-through stub.

   The Problem in Agentic Workflows:
   In multi-step autonomous pipelines, PII (emails, phone numbers, SSNs, credit cards, physical addresses) rarely lives in predictable key names. It appears inside unstructured text payloads (eg. customer support transcripts, scraped docs, user queries, SQL query parameters). If Step 1 retrieves customer records and Step 2 calls an untrusted third-party tool or cloud LLM, unmasked PII leaks across security boundaries and pollutes the audit spine.

   ---

   Recommended Architecture: Two-Tier Pluggable Engine

   AURA is an architectural "runtime coat" designed to be lightweight, fast, and host-agnostic (dependencies = ["pyyaml"]). We should not hard-code a 270M parameter Transformer model (eg. ARPA's own micro-f1-mask) or heavy NLP framework directly into AURA's base dependencies.

   Instead, we should adopt a two-tier architecture:

                  Sequencer Step Inbound / Outbound Context
                                     │
                                     ▼
                        ┌──────────────────────────┐
                        │   middleware.pii_mask    │
                        └────────────┬─────────────┘
                                     │
               ┌─────────────────────┴─────────────────────┐
               ▼                                           ▼
   ┌───────────────────────────┐               ┌───────────────────────────┐
   │ Tier 1: Built-in Engine   │               │ Tier 2: Pluggable Adapter │
   │ (Pure Python / Stdlib)    │               │ (Model / Skillware / NER) │
   ├───────────────────────────┤               ├───────────────────────────┤
   │ • Zero dependencies       │               │ • PiiMaskerProvider hook  │
   │ • Regex + Luhn validation │               │ • ARPA micro-f1-mask 270M │
   │ • Emails, phones, SSNs,   │               │ • Skillware pii_masker    │
   │   IPs, cards, keys, UUIDs │               │ • Contextual entities     │
   │ • Microsecond execution   │               │   (names, addresses, org) │
   └───────────────────────────┘               └───────────────────────────┘
                                     │
                                     ▼
                  Transform Strategy (Redact | Mask | Pseudonymize)
                                     │
                                     ├─► Emit 'middleware.applied' on Spine
                                     │   (entity counts, 0 leaked PII)
                                     ▼
                  Safe Step Execution / Clean Audit Spine
   

   Tier 1: Built-in Engine (NativePiiMasker)

   • Zero dependencies (Python standard library re).
   • High-speed pattern detection with algorithmic verification:
     • Email: RFC-compliant pattern.
     • Phone Numbers: E.164 and international/domestic standard formats.
     • Credit Cards: Format matching + Luhn checksum algorithm validation to avoid false positives on arbitrary 16-digit numbers.
     • SSN / National IDs: Standard regex pattern checks.
     • IP Addresses: IPv4 and IPv6 format validators.
     • Secrets & Keys: High-entropy strings, JWTs, UUIDs, bearer token patterns.

   Tier 2: Pluggable Provider Protocol (PiiMaskerProvider)

   • For enterprise and regulated environments requiring contextual Named Entity Recognition (names of persons, organizations, medical terms, physical addresses).
   • An explicit protocol:

     class PiiMaskerProvider(Protocol):
         def mask_text(self, text: str, config: PiiMaskConfig) -> PiiMaskResult: ...

   • Integrates seamlessly with:
     • Skillware: Using Skillware's pii_masker skill or host tool.
     • ARPA micro-f1-mask: A local/remote adapter connecting to the 270M parameter model when running in an environment with ML extras installed.
     • External Engines: Microsoft Presidio, AWS Comprehend, or internal enterprise microservices.
   • Fail-safe posture: If Tier 2 fails or is unconfigured, fallback gracefully to Tier 1 regex or fail-closed based on on_error: fallback | fail | pass.

   ---

   Three Core Masking Strategies

   Industry best practice (NIST SP 800-122 & GDPR) dictates that masking cannot be one-size-fits-all. AURA should support three strategies:

   1. redact (Default): Complete replacement with entity tags:
      • "Contact john@corp.com at 555-0199" → "Contact [EMAIL] at [PHONE]"
   2. mask (Partial obfuscation for operational legibility):
      • "john.doe@corp.com" → "j***e@corp.com", "4111111111111234" → "****-****-****-1234"
   3. pseudonymize / tokenize (Essential for Agentic Reasoning):
      • If an agent is executing a multi-step investigation on customer alice@corp.com, redacting to generic [EMAIL] destroys the LLM's ability to correlate Alice across Step 2 and Step 3.
      • Pseudonymization creates a session-scoped HMAC token (e.g. <EMAIL_REF_a1b2>), preserving entity equivalence across steps without revealing the true plaintext PII!

   ---

   Pipeline Invocation Models

   middleware.pii_mask should be usable in two complementary ways:

   A. Automatic Boundary Middleware (scope: per_step)

   Configured on the sequencer policy to automatically sanitize data moving between steps:

   sequencer:
     middleware:
       scope: per_step
       order:
         - op: middleware.pii_mask
           config:
             strategy: pseudonymize
             entities: ["email", "phone", "credit_card", "ssn"]
             target: "args" # sanitize inbound step arguments
     steps:
       - id: fetch_user
         type: skill
         ref: user_service
       - id: analyze_data
         type: skill
         ref: analytics_tool

   B. Explicit Sequencer Pipeline Step (type: op)

   Used when a pipeline wants a discrete, audited transformation step in its DAG:

   steps:
     - id: ingest_raw_data
       type: skill
       ref: db_query
     - id: sanitize_step
       type: op
       ref: middleware.pii_mask
       config:
         source_step: ingest_raw_data
         strategy: redact
         entities: ["all"]
     - id: public_llm_summary
       type: prompt
       ref: summarize_clean_data

   ---

   Audit Spine Contract & Telemetry

   AURA's core rule: The audit spine must never record the raw secret or PII.

   When masking occurs, emit middleware.applied (or sequencer.middleware.applied):

   {
     "op": "middleware.pii_mask",
     "step_id": "sanitize_step",
     "provider": "native_regex",
     "strategy": "redact",
     "entities_detected": {
       "email": 2,
       "phone": 1,
       "credit_card": 0
     },
     "total_redactions": 3,
     "fields_modified": ["args.user_text", "args.query"]
   }

   • Spectrum Posture:
     • At spectrum.level: low / mid: Redact/mask and proceed with middleware.applied.
     • At spectrum.level: high / full: Configurable strict: true mode can emit constraint.violated / finding: PII_DETECTED if sensitive PII is encountered in an unapproved boundary.

   ---

   Proposed Scope of Work & Deliverables

   1. aura/sequencer/pii_mask.py:
      • NativePiiMasker: Stdlib regex + Luhn-validated engine supporting redact, mask, and pseudonymize.
      • PiiMaskerProvider protocol for pluggable backends.
      • Deep traversal helper to mask nested dicts, lists, and string payloads.
   2. aura/sequencer/middleware.py:
      • Wire MiddlewareStack._invoke to dispatch registered ops (starting with middleware.pii_mask).
      • Execute inbound/outbound hooks in SequencerRunner._run_step.
   3. aura/sequencer/runner.py:
      • Support type: op, ref: middleware.pii_mask execution in StepBackend.run_op.
      • Emit middleware.applied on the session spine with metadata only (zero PII leakage).
   4. Integration Adapter (Optional Extra):
      • integrations/skillware/pii_adapter.py: Bridge to Skillware pii_masker / micro-f1-mask if installed.
   5. Tests & Demos:
      • tests/test_sequencer_pii_mask.py: Unit tests for regex, Luhn check, strategies, nested data, and spine telemetry.
      • Example pipeline in examples/06-skillware-sequencer-chain or examples/04-sequencer-pipeline.
      • Updated documentation in docs/sequencer.md and docs/outputs.md.
5. rosspeili

   self-assigned this

   on Oct 11, 2026
6. rosspeili

   added a commit that references this issue on Oct 11, 2026

   feat(sequencer): middleware.pii_mask op and boundary stack (#58) (#90)

   ...

   0e1a96f
7. rosspeili commented on Oct 15, 2026

   

   rosspeili

   on Oct 15, 2026

   ContributorAuthor

   More actions

   closing as done by #90