Rehabilitate legacy endpoint lists as privacy-preserving Netveil audit

[omar07ibrahim]

Scope

This draft rehabilitates an old endpoint-list repository as Netveil in incremental, reviewable layers.

Commit 1 — remove unsafe legacy data

• removes unverified third-party IP/port inventories from the current tree
• documents that historical values do not grant permission to connect or test
• adds an explicit data-handling and responsible-disclosure policy

Commit 2 — fail-closed offline parser

• parses exact local bytes with strict UTF-8, line-ending, IP, port, and resource bounds
• canonicalizes IPv4 and bracketed IPv6 endpoints and detects equivalent duplicates
• returns redacted errors and binds the private corpus model to exact input bytes
• blocks impossible public model construction and ships typed installed artifacts

Commit 3 — keyed aggregate privacy receipt

• produces a public aggregate with no raw endpoint, canonical endpoint, secret key, or unkeyed source digest
• uses separately domain-framed HMAC-SHA256 IDs for exact source content and duplicate endpoint groups
• emits endpoint-occurrence counts by IP version, scope, and fixed port range
• canonicalizes public JSON and binds it with a non-self-referential SHA-256 report digest
• embeds the exact Python runtime profile because stdlib ipaddress semantics can differ between interpreter releases
• documents the byte-level protocol, typed prefixes, framing, aggregation semantics, and threat boundary

Commit 4 — exception-chain privacy hardening

• prevents rejected UTF-8 payloads and invalid addresses from surviving in library-created exception context
• verifies bounded public errors across repr, cause/context, and formatted traceback surfaces
• documents the exact boundary: caller objects, ambient context, frame locals, debuggers, and crash dumps remain caller-controlled

Privacy and safety boundary

This is pseudonymization, not anonymity. A reused key enables equality linkage; aggregate sizes and duplicate frequency remain visible; a chosen-input service acts as an online guessing oracle. HMAC only resists offline enumeration while a high-entropy key stays secret. Python does not guarantee zeroization. The report digest is integrity metadata, not a signature or proof of origin.

The package performs no DNS resolution, socket operation, network probe, file access, or child-process execution. Parser models still contain raw canonical addresses and an unkeyed source SHA, so they remain private-side objects.

The removed endpoint strings remain in Git history until a separately coordinated history rewrite. No history rewrite is attempted here.

Validation

• 32 parser/model/packaging/privacy tests pass
• 100% statement and branch coverage across 439 package statements and 154 branches
• Ruff check and format check pass
• strict mypy passes
• known-answer HMAC vectors bind the exact domain and length framing
• key rotation, domain separation, equivalent IPv6 grouping, deterministic ordering, impossible-state, redaction, and resource-bound tests pass
• sdist and wheel build; the sdist contains SECURITY and the protocol document
• the wheel installs into a clean environment and all 32 tests pass against the installed artifact
• netveil/py.typed is present and the installed dependency set is consistent
• source scan finds no networking or process API in the package
• all four commits have Omar Ibrahim as author and committer

Next draft layers

• offline CLI with safe key-file handling and installed-artifact guards
• real source-derived CLI output capture and architecture/workflow diagrams
• reproducible demo recording generated from the shipped CLI

The PR intentionally remains a draft until those layers are implemented and independently reviewed.