From c4d831ca4006b452d9641a6b6fe3aa2673924de1 Mon Sep 17 00:00:00 2001 From: Karan Bansal Date: Sat, 3 Oct 2026 19:56:38 +0530 Subject: [PATCH 1/3] Harden the rules, add the public false-positive corpus, sync docs to the live run --- README.md | 82 ++- plugins/redact/DECISIONS.md | 8 +- plugins/redact/README.md | 39 +- plugins/redact/RULES.md | 20 +- plugins/redact/hooks/register.ts | 8 +- plugins/redact/hooks/rules.ts | 169 +++++- plugins/redact/tests/register.test.ts | 10 +- plugins/redact/tests/rules.test.ts | 184 +++++- site/index.html | 22 +- tools/fp-corpus-labels.json | 410 +++++++++++++ tools/fp-corpus-results.md | 805 ++++++++++++++++++++++++++ tools/fp-corpus.mjs | 347 +++++++++++ 12 files changed, 2018 insertions(+), 86 deletions(-) create mode 100644 tools/fp-corpus-labels.json create mode 100644 tools/fp-corpus-results.md create mode 100644 tools/fp-corpus.mjs diff --git a/README.md b/README.md index b2e6836..0c58540 100644 --- a/README.md +++ b/README.md @@ -2,22 +2,27 @@ [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](#license) -**Every key a developer pastes into Claude Code goes to the model and stays in a session file on disk. redact swaps it for a placeholder inside Claude Code, before the model reads it and before the transcript keeps it, with zero network reach.** +**Every key a developer pastes into Claude Code goes to the model and stays in a session file on disk. redact swaps it for a placeholder inside Claude Code before the model reads it, keeps the placeholder in the conversation rows the session file stores, and puts the real value back only inside file edits. Zero network reach.** Site: [redact.aidojo.si](https://redact.aidojo.si). It is a Claude Code [mod](https://code.claude.com/docs/en/plugins/mods/overview): a plugin of TypeScript function hooks that runs in the Claude Code process. ## What happens ``` -# Headless `claude -p` run on Claude Code 2.1.288. The prompt held a fake AWS access key id. -# Typed (key masked here): -my aws key is AKIA**************** please remember it -# The user row as stored in the session file: -my aws key is [REDACTED:AWS_KEY#20fae26c] please remember it -# The stored system-prompt snapshot carried the redaction notice that tells Claude the placeholder is opaque. +# Interactive session on Claude Code 2.1.288, Opus 5.5, accept-edits mode. Fake key, masked here as AKIA****************. +⏺ redact: hid 1 AWS_KEY in your prompt +❯ Save this AWS access key id to a new file named key.txt in this directory, exactly as given, then stop: [REDACTED:AWS_KEY#08956cf3] +⏺ Write(key.txt) + ⎿ Wrote 1 line to key.txt + 1 AKIA**************** +⏺ I saved the AWS access key ID to key.txt, exactly as you gave it with no trailing newline. The write tool fills in the + real value when it creates the file, so it should be on disk as you sent it. I didn't open the file to check, because that would print the key back + into this conversation. ``` -The same value gets the same placeholder for the whole session. When Claude later edits a file that holds it, redact puts the real value back inside the `Edit`, `Write` or `NotebookEdit` arguments only. +`key.txt` on disk held the real key. The session file held the placeholder in every conversation row; `/redact` answered `hid 1 values this session (1 AWS_KEY); restored 1 placeholders`. + +The same value gets the same placeholder for the whole session. When Claude later edits a file that holds it, redact puts the real value back inside the `Edit`, `Write` or `NotebookEdit` arguments only. In a live session this was driven end to end: a key typed in the prompt, Claude asked to save it to a file, the file on disk holding the real key while every conversation row holds the placeholder. See [Where it was proven](#where-it-was-proven). ## Install @@ -35,24 +40,22 @@ To try it for one session without installing: `claude --plugin-dir ./plugins/red - **Reach L0.** It runs no process, reads no file, env var or setting, and writes no file. The validator lines in [What it reaches](#what-it-reaches) list every hook, call and state key. - **No network.** Nothing leaves the machine. It starts no daemon and calls no scanner binary. - **No model call.** Detection never asks a model. A redactor that sent your text to a model to find secrets would defeat its own purpose. -- **Deterministic rules.** 23 secret rules and 7 opt-in PII rules: regex plus checksums (Luhn, Verhoeff, IBAN mod-97, JWT header decode). The same input gives the same output. Every quantifier is bounded. +- **Deterministic rules.** 24 secret rules and 7 opt-in PII rules: regex plus checksums (Luhn, Verhoeff, IBAN mod-97, JWT header decode). The same input gives the same output. Every quantifier is bounded. - **Reversible only inside `Edit`, `Write` and `NotebookEdit`.** Placeholders turn back into real values only in those arguments, so an edit to a line that held a key still matches the file. `Bash` never gets the real value, so the model cannot echo a placeholder into a network command and send the secret out. - **Fail closed.** A row that makes the scanner throw is replaced with `[redact withheld this content: the redactor failed while scanning it]`. It is never stored raw. - **The transcript file too.** It rewrites at `session.append`, the one event that sees every row before it is stored and sent: prompts, tool results, Claude's own blocks, attachments, notices. The model and the session file get the same rewritten row. - **Proof files.** The repo ships the threat model below, [`RULES.md`](plugins/redact/RULES.md) with a public source for every rule, and [`DECISIONS.md`](plugins/redact/DECISIONS.md) with every rejected option and every assumption not yet checked. -## Proof so far - -From a run on Claude Code 2.1.288 with the mod-builder harness: - -- `claude plugin validate` passed. -- The module loaded with events `session.start`, `prompt.submit`, `session.append`, `tool.call`, `prompt.section`, `command.run`. -- `tsc` passed. -- 60 kit tests pass (`claude plugin test`). -- `/redact` answered in a headless run. -- A headless `claude -p` run with a fake AWS key stored the user row as shown in [What happens](#what-happens), and the stored system-prompt snapshot carried the redaction notice. +## Where it was proven -Not done yet: a live interactive run with a logged-in session and a real file edit by the model. The kit tests cover the argument rewrite in `Edit`, `Write` and `NotebookEdit`. A real edit end to end has not been run. +| Surface | Status | Evidence | +|---|---|---| +| Terminal, interactive, logged in | ran and passed | A session on Claude Code 2.1.288 (Opus 5.5): the prompt echo showed the placeholder, Claude's `Write` call carried the placeholder, the file on disk holds the real key, the session file holds the placeholder in every conversation row and `/redact` reported `restored 1 placeholders` | +| Headless `claude -p` | ran and passed | The stored user row held the placeholder; the system-prompt snapshot carried the redaction notice; `/redact` answered | +| Plugin test kit | ran and passed | `claude plugin test`: 60 pass, 0 fail, on the real engine | +| Subagent rows | unverified | The types say `session.append` and `tool.call` fire for subagents with `agentId`; not yet driven live | +| `/reload-plugins` | unverified | Designed for: the value map lives in `$.state`, which survives a reload | +| SDK, VS Code panel, Desktop, cloud sessions | unverified | Same hooks, no drawing; not yet run there | ## What it reaches @@ -92,12 +95,28 @@ The same secret gets the same placeholder across rows, edits and compaction summ ## What it catches -Secrets, always on: AWS access keys and secret keys, GitHub, GitLab, Slack tokens and webhooks, Stripe live keys, SendGrid, Twilio, Google API keys and OAuth secrets, Anthropic, OpenAI, Hugging Face, npm and PyPI tokens, Azure storage keys, JWTs, PEM private keys, database URLs with a password, `Authorization` header values, and generic `password=` / `api_key=` style assignments. +Secrets, always on: AWS access keys and secret keys, GitHub, GitLab, Slack tokens and webhooks, Stripe live keys, SendGrid, Twilio, Google API keys and OAuth secrets, Anthropic, OpenAI, Hugging Face, npm and PyPI tokens, Azure storage keys, JWTs, PEM private keys (complete or truncated), database URLs with a password, `Authorization` header values, generic `password=` / `api_key=` style assignments, and any of the prefix-shaped tokens above wrapped in base64. PII, opt-in with `pii: true`: email, payment cards (Luhn), India Aadhaar (Verhoeff), US SSN, IBAN (mod-97), India PAN, phone numbers. Every rule, with its id, a fake example and its source: [`RULES.md`](plugins/redact/RULES.md). +## False positives on ordinary code + +A redactor that fires on normal work gets uninstalled in a week, so the number to defend is the false-positive count, not the catch count. `tools/fp-corpus.mjs` shallow-fetches 11 public repositories at pinned commits (lockfiles, Go sums, Terraform, Markdown, minified JS, JWT and PEM test vectors, a dotenv test suite) and scans every text file with the secret rules: + +| Files scanned | Secret-rule hits | Real test keys and vectors | Fake fixtures | False positives | +|---|---|---|---|---| +| 42,836 | 68 | 60 | 8 | 0 | + +Every hit was opened at its line and labelled by hand; the labels live in `tools/fp-corpus-labels.json`, so a rerun that produces a new hit shows it as unreviewed instead of counting it. Rerun it yourself: + +``` +node tools/fp-corpus.mjs +``` + +The full per-hit list is in [`tools/fp-corpus-results.md`](tools/fp-corpus-results.md). The 60 real-shaped hits are complete private keys and signed JWTs published as test vectors, and demo database URLs with a password; a redactor should hide those. The PII rules, off by default, hit 650 times on the same corpus (mostly email addresses in docs), which is why they are opt-in. + ## Options Stored under `pluginConfigs` in settings. A change reloads the mod. @@ -108,17 +127,16 @@ Stored under `pluginConfigs` in settings. A change reloads the mod. ## Limitations -- Two engine bookkeeping records in the session file are stored as made and can keep the typed prompt: the `queue-operation` record written at enqueue, and a command's `args` stamp. Neither is sent to the model. In the headless `-p` run above, both still held the typed text. In the headless path the queue record is written before `prompt.submit` runs. This is a known gap. -- A secret the rules do not know stays visible. Generic `key=value` detection needs the value to look like a secret (length and entropy): `password=hunter2` is caught, `token=abc` is not. -- A secret split across two rows, or shown partially (the last four characters), is not caught. -- Images are not scanned. A screenshot of a key goes through. -- Claude cannot search for a secret by value: `Grep` for a placeholder finds nothing. Search by the line around it. -- Plugins loaded ahead of this one see the row before it is rewritten. -- The screen may show a row just before its rewrite. The model and the session file never see that form. -- Nothing is drawn in `claude -p`, the SDK, the VS Code panel or cloud sessions. The `redact: hid ...` line arrives as `ui_log` there. `/redact` still reports totals; it answered in the headless run. -- A pattern guard is not a permission rule. A determined prompt can describe a secret in words the rules do not match. - -Assumptions not yet checked (subagent rows, restore after `/reload-plugins`, the interactive queue record) are listed with a way to check each one in [`DECISIONS.md`](plugins/redact/DECISIONS.md). +- **Three host bookkeeping records are stored as made and can hold a raw value.** The engine lets a mod rewrite the conversation rows the model reads, not the records around them. (a) The `toolUseResult` record of a `Write`, `Edit` or `NotebookEdit` call keeps the file content as written, so a value restored into a file lands there in clear. (b) A slash command's `args` stamp keeps the typed arguments. (c) In headless `claude -p` only, the `queue-operation` record keeps the typed prompt, because it is written before `prompt.submit` runs; an interactive session writes no such record. None of the three is sent to the model. +- **`tool_use` blocks are not rewritable.** The engine puts the model's own tool call blocks back as made; a mod may rewrite text blocks and tool results only. The model only ever sees placeholders, so a raw value in a tool call means it came from text the rules missed. +- **A secret the rules do not know stays visible.** Generic `key=value` detection needs the value to look like a secret (length and entropy): `password=hunter2` is caught, `token=abc` is not. +- **A secret split across two rows, or shown partially** (the last four characters), is not caught. +- **Images are not scanned.** A screenshot of a key goes through. +- **Claude cannot search for a secret by value.** `Grep` for a placeholder finds nothing. Search by the line around it. +- **Plugins loaded ahead of this one** see the row before it is rewritten. +- **The screen may show a row just before its rewrite.** The model and the session file never see that form. +- **Nothing is drawn in `claude -p`, the SDK, the VS Code panel or cloud sessions.** The `redact: hid ...` line arrives as `ui_log` there. `/redact` works everywhere. +- **A pattern guard is not a permission rule.** A determined prompt can describe a secret in words the rules do not match. ## For teams diff --git a/plugins/redact/DECISIONS.md b/plugins/redact/DECISIONS.md index 70303bc..789cbd9 100644 --- a/plugins/redact/DECISIONS.md +++ b/plugins/redact/DECISIONS.md @@ -12,9 +12,11 @@ - No `turn.complete` line and no `$.ui.notice`. Rejected: a per-turn summary, because the per-row `$.ui.log` line already says what was hidden and where; `$.ui.notice` is removed when the tool call resolves, which is before the result row is appended. - The scrub logic is a pure module (`hooks/scrub.ts`) tested directly. Rejected: testing it through `session.append` in the kit, because on Claude Code 2.1.288 a test stub for `session.append` cannot answer (every answer must follow a `next`, and nothing is beneath the stub). +## Verified since +- The `$.ui.log` line draws in the terminal transcript as `redact: hid 1 AWS_KEY in your prompt`; the engine prefixes the plugin name, so the mod's own text carries none. +- Live interactive session on 2.1.288: the prompt echo showed the placeholder, Claude's `Write` call carried it, the file on disk holds the real value, `/redact` reported one restoration. The session file held the placeholder in every conversation row; the only raw copy was the `toolUseResult` record of the `Write` (the file content as written), which the engine stores as made. +- An interactive session writes no `queue-operation` record at all; only the headless `-p` path does, and there it is written before `prompt.submit` runs. + ## Assumptions not verified -- In an interactive session, `prompt.submit` runs before the `queue-operation` record is written, so that record holds the placeholder. In the headless `-p` path it is written first and holds the typed text. If wrong: the typed prompt stays raw in that one bookkeeping record (never sent to the model). How to check: type a prompt with a fake key in an interactive session and grep the session file for `queue-operation`. - Subagent rows and subagent `Edit` calls pass through the same hooks with `agentId` set, as the types say. If wrong: a subagent could read a raw secret. How to check: run a subagent that reads a file with a fake key and grep its rows in the session file. -- The restore path works against a real file edit by the model. The kit proves the argument rewrite; a live run with a logged-in session has not been done. How to check: ask Claude to move a key from `.env` to another file and confirm the new file holds the real value. - `$.state` survives `/reload-plugins`, so placeholders made before a reload still restore. If wrong: an edit using an older placeholder fails and Claude re-reads the file. How to check: hide a value, run `/reload-plugins`, ask for an edit of that line. -- The `$.ui.log` line draws in the terminal transcript. The `-p` path receives it as `ui_log`. If wrong: the person loses the per-row notice; `/redact` still reports totals. How to check: an interactive session with a fake key in the prompt. diff --git a/plugins/redact/README.md b/plugins/redact/README.md index ef2e3b7..bfa63c7 100644 --- a/plugins/redact/README.md +++ b/plugins/redact/README.md @@ -2,7 +2,7 @@ Redacts secrets and PII from every row Claude Code stores, before the model reads it and before the transcript keeps it, and puts the real value back only inside `Edit`, `Write` and `NotebookEdit` arguments so file edits still work. -Built on Claude Code 2.1.288. Proven on 2.1.288 (stages: validate, load, typecheck, test, command, isolation). Requires Claude Code 2.1.287 or later. +Built on Claude Code 2.1.288. Proven on 2.1.288 (stages: validate, load, typecheck, test, command, isolation, plus a live interactive session with a real file write). Requires Claude Code 2.1.287 or later. ## What it can reach @@ -32,7 +32,23 @@ Threat model for redact (reach L0, draws and remembers) The hash is FNV-1a over a per-session salt plus the value, so the same secret gets the same placeholder across rows, edits and compaction summaries, and a placeholder from another session means nothing here. -Detection is deterministic: 23 secret rules and 7 PII rules, regex plus checksums (Luhn, Verhoeff, IBAN mod-97, JWT header decode), no model call. The list with sources is in `RULES.md`. +Detection is deterministic: 24 secret rules and 7 PII rules, regex plus checksums (Luhn, Verhoeff, IBAN mod-97, JWT header decode), no model call. The list with sources is in `RULES.md`. + +## False positives on ordinary code + +A redactor that fires on normal work gets uninstalled in a week, so the number to defend is the false-positive count, not the catch count. `tools/fp-corpus.mjs` shallow-fetches 11 public repositories at pinned commits (lockfiles, Go sums, Terraform, Markdown, minified JS, JWT and PEM test vectors, a dotenv test suite) and scans every text file with the secret rules: + +| Files scanned | Secret-rule hits | Real test keys and vectors | Fake fixtures | False positives | +|---|---|---|---|---| +| 42,836 | 68 | 60 | 8 | 0 | + +Every hit was opened at its line and labelled by hand; the labels live in `tools/fp-corpus-labels.json`, so a rerun that produces a new hit shows it as unreviewed instead of counting it. Rerun it yourself: + +``` +node tools/fp-corpus.mjs +``` + +The full per-hit list is in `tools/fp-corpus-results.md` at the repo root. The 60 real-shaped hits are complete private keys and signed JWTs published as test vectors, and demo database URLs with a password; a redactor should hide those. The PII rules, off by default, hit 650 times on the same corpus (mostly email addresses in docs), which is why they are opt-in. ## Install @@ -55,15 +71,16 @@ Stored under `pluginConfigs` in settings; a change reloads the mod. ## Limitations -- Two engine bookkeeping records in the session file are stored as made and can keep the typed prompt: the `queue-operation` record written at enqueue, and a command's `args` stamp. Neither is sent to the model. In the headless `-p` path the queue record is written before `prompt.submit` runs. -- A secret the rules do not know stays visible. Generic `key=value` detection needs the value to look like a secret (length and entropy); `password=hunter2` is caught, `token=abc` is not. -- A secret split across two rows, or shown partially (the last four characters), is not caught. -- Images are not scanned. A screenshot of a key goes through. -- Claude cannot search for a secret by value: `Grep` for a placeholder finds nothing. Search by the line around it. -- Plugins loaded ahead of this one see the row before it is rewritten. -- The screen may show a row just before its rewrite; the model and the session file never see that form. -- Nothing is drawn in `claude -p`, the SDK, the VS Code panel or cloud sessions; the `redact: hid ...` line arrives as `ui_log` there, and `/redact` works everywhere. -- A pattern guard is not a permission rule. A determined prompt can describe a secret in words the rules do not match. +- **Three host bookkeeping records are stored as made and can hold a raw value.** The engine lets a mod rewrite the conversation rows the model reads, not the records around them. (a) The `toolUseResult` record of a `Write`, `Edit` or `NotebookEdit` call keeps the file content as written, so a value restored into a file lands there in clear. (b) A slash command's `args` stamp keeps the typed arguments. (c) In headless `claude -p` only, the `queue-operation` record keeps the typed prompt, because it is written before `prompt.submit` runs; an interactive session writes no such record. None of the three is sent to the model. +- **`tool_use` blocks are not rewritable.** The engine puts the model's own tool call blocks back as made; a mod may rewrite text blocks and tool results only. The model only ever sees placeholders, so a raw value in a tool call means it came from text the rules missed. +- **A secret the rules do not know stays visible.** Generic `key=value` detection needs the value to look like a secret (length and entropy): `password=hunter2` is caught, `token=abc` is not. +- **A secret split across two rows, or shown partially** (the last four characters), is not caught. +- **Images are not scanned.** A screenshot of a key goes through. +- **Claude cannot search for a secret by value.** `Grep` for a placeholder finds nothing. Search by the line around it. +- **Plugins loaded ahead of this one** see the row before it is rewritten. +- **The screen may show a row just before its rewrite.** The model and the session file never see that form. +- **Nothing is drawn in `claude -p`, the SDK, the VS Code panel or cloud sessions.** The `redact: hid ...` line arrives as `ui_log` there. `/redact` works everywhere. +- **A pattern guard is not a permission rule.** A determined prompt can describe a secret in words the rules do not match. ## Uninstall diff --git a/plugins/redact/RULES.md b/plugins/redact/RULES.md index abe1c53..f11c7c7 100644 --- a/plugins/redact/RULES.md +++ b/plugins/redact/RULES.md @@ -7,7 +7,8 @@ How a scan settles conflicts: - A hit covers only the value (the capture group), not the key name or quotes around it. - When two hits overlap, the leftmost wins, then the longest, then the rule listed first below. - Text inside an existing `[REDACTED:LABEL#hex]` placeholder is never matched again. -- Every quantifier is bounded, so a 2 MB blob of random base64 scans in well under a second. +- A secret wrapped in base64 is reported once, over the whole base64 run, with the label of the rule that found it inside (see below). +- Every quantifier is bounded, and so is the base64 decode pass, so a 2 MB blob of random base64 or of near-miss keys scans in well under a second. All examples below are fake. They have the right shape and nothing else. @@ -33,11 +34,20 @@ All examples below are fake. They have the right shape and nothing else. | `pypi-token` | `PYPI_TOKEN` | secret | PyPI upload token: `pypi-AgEIcHlwaS5vcmc` plus at least 50 characters | `pypi-AgEIcHlwaS5vcmcFAKE0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKL` | [gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | | `azure-storage-key` | `AZURE_STORAGE_KEY` | secret | Azure storage account key (86 base64 characters plus `==`) after `AccountKey=` or a storage key name | `AccountKey=FAKEfake0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz+/FAKEfake012345==` | [Microsoft Learn](https://learn.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string) | | `jwt` | `JWT` | secret | JSON Web Token: three base64url segments whose header decodes to JSON with an `alg` field | `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IlRlc3QgVXNlciIsImlhdCI6MTUxNjIzOTAyMn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c` | [RFC 7519](https://datatracker.ietf.org/doc/html/rfc7519) | -| `private-key` | `PRIVATE_KEY` | secret | Whole PEM private key block (RSA, EC, DSA, OPENSSH, ENCRYPTED, PGP), across lines | `-----BEGIN RSA PRIVATE KEY-----MIIEFAKEFAKEFAKE0123-----END RSA PRIVATE KEY-----` | [gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | -| `db-connection-url` | `DB_URL` | secret | postgres, postgresql, mysql, mariadb, mongodb, mongodb+srv, redis, rediss, amqp and amqps URLs that carry `user:password@`; placeholder passwords are skipped | `postgres://admin:FakePassw0rd1@db.example.test:5432/app` | [GitHub secret scanning patterns](https://docs.github.com/en/code-security/secret-scanning/introduction/supported-secret-scanning-patterns) | +| `private-key` | `PRIVATE_KEY` | secret | Whole PEM private key block (RSA, EC, DSA, OPENSSH, ENCRYPTED, PGP), across lines, with a body of up to 65536 characters | `-----BEGIN RSA PRIVATE KEY-----MIIEFAKEFAKEFAKE0123-----END RSA PRIVATE KEY-----` | [gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | +| `private-key-truncated` | `PRIVATE_KEY` | secret | A private key BEGIN line with no END line, such as the first lines of a key pasted from a terminal: the marker, any `Proc-Type:`, `DEK-Info:` or `Version:` header lines, then at least two base64 lines of 16 to 128 characters, up to the end of the run (at most 2048 lines). Lines may be indented and split by real or JSON-escaped (`\n`) newlines. When the END line is there, `private-key` wins because its hit is longer | `-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANFAKEfakeFAKEfake0123\nFAKEfakeFAKEfake0123456789abcdef` | [RFC 7468](https://www.rfc-editor.org/rfc/rfc7468) | +| `db-connection-url` | `DB_URL` | secret | postgres, postgresql, mysql, mariadb, mongodb, mongodb+srv, redis, rediss, amqp and amqps URLs that carry `user:password@`; placeholder passwords (`password`, `pass`, `%s`, `%d`, `{{...}}`, `<...>`, `${...}`) and URLs wrapped in `{{ }}` (docs templates) are skipped | `postgres://admin:FakePassw0rd1@db.example.test:5432/app` | [GitHub secret scanning patterns](https://docs.github.com/en/code-security/secret-scanning/introduction/supported-secret-scanning-patterns) | | `auth-header` | `AUTH_HEADER` | secret | Value of an `Authorization: Bearer`, `Basic` or `Token` header | `Authorization: Bearer FAKE0123456789abcdef` | [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110#name-authorization) | -| `generic-secret-quoted` | `SECRET` | secret | Quoted value of 8+ characters assigned to a name containing api key, secret, token, password, passwd, pwd, auth or credential | `password: "Fake-Passw0rd!"` | [gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | -| `generic-secret` | `SECRET` | secret | Same names with a bare value; skips booleans, placeholders, env references, paths, code identifiers and values under 3.0 bits per character of entropy | `API_KEY=fake0123456789abcdef` | [gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | +| `generic-secret-quoted` | `SECRET` | secret | Quoted value of 8+ characters assigned to a name containing api key, secret, token, password, passwd, pwd, auth or credential. The name must start a word: after a non-alphanumeric character other than `:`, after a literal `\n`, or as a camelCase hump (`clientSecret`), so `oauth:`, `OAuth` and `force:auth:` do not count. Template slots (`{{...}}`, `<...>`, `__NAME__`, `%s`, `Name=`) are skipped | `password: "Fake-Passw0rd!"` | [gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | +| `generic-secret` | `SECRET` | secret | Same names and word-start rule with a bare value, which ends at whitespace or a literal `\n`, `\r` or `\t`; skips booleans, placeholders, env references, paths, values under 3.0 bits per character of entropy and code: member chains (`jose.base64url.decode`), optional chains (`options?.auth`) and type names (`Uint8Array`) | `API_KEY=fake0123456789abcdef` | [gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | + +## Base64-wrapped secrets + +After the rules above, `scan()` makes one decode pass. It takes up to 200 runs of the base64 alphabet (standard or url-safe, not mixed) of 24 to 8192 characters, padded to a multiple of 4 or unpadded url-safe. It decodes each one and keeps it only when at least 90% of the decoded bytes are printable ASCII. Then it runs only the prefix-anchored rules on the decoded text: `aws-access-key`, `github-token`, `gitlab-pat`, `slack-token`, `slack-webhook`, `stripe-live-key`, `sendgrid-key`, `twilio-key`, `google-api-key`, `google-oauth-secret`, `anthropic-key`, `openai-key`, `huggingface-token`, `npm-token`, `pypi-token`, `private-key`, `private-key-truncated`, `db-connection-url` and `jwt`. The generic and PII rules never run on decoded text, because decoded prose would trip them. Images, lockfile integrity hashes and random blobs decode to binary and produce nothing. A JWT is three separate base64url runs, so the `jwt` hit over the whole token wins. + +| id | label | kind | what it catches | example that matches | source | +| --- | --- | --- | --- | --- | --- | +| `-base64`, for example `aws-access-key-base64` | label of the inner rule | secret | One hit over the whole base64 run when the first inner rule, in table order, matches the decoded text. `off` with `aws-access-key-base64` skips only the wrapped form; `off` with `aws-access-key` skips both | `QUtJQUZBS0UyMzQ1VEVTVDY3Njc=` (the AWS example above, run through `base64`) | [RFC 4648](https://www.rfc-editor.org/rfc/rfc4648) | ## PII rules (opt-in) diff --git a/plugins/redact/hooks/register.ts b/plugins/redact/hooks/register.ts index 4473a15..7a26932 100644 --- a/plugins/redact/hooks/register.ts +++ b/plugins/redact/hooks/register.ts @@ -48,7 +48,7 @@ async function remember($: EngineInterface, pairs: ReadonlyArray { return next({ ...e, message: { ...e.message, content: r.blocks } }) }).catch(($, e, next) => { if (next.called) return next(e) - $.ui.log(`redact: scanning failed, so this row was withheld from the model (${where(e)})`) + $.ui.log(`scanning failed, so this row was withheld from the model (${where(e)})`) return next({ ...e, message: { ...e.message, content: withhold(e.message.content) } }) }) @@ -122,7 +122,7 @@ export const register: Register = (on, options) => { const k = await read($, restored) const total = Object.values(c).reduce((a, b) => a + b, 0) const rules = cfg.pii ? 'secrets and PII' : 'secrets only (set pii to true for email, phone, cards, national ids)' - if (total === 0) return { text: `redact: nothing hidden yet this session. Scanning ${rules}.` } - return { text: `redact: hid ${total} values this session (${describe(c)}); restored ${k} placeholders inside Edit, Write and NotebookEdit arguments. Scanning ${rules}.` } + if (total === 0) return { text: `nothing hidden yet this session. Scanning ${rules}.` } + return { text: `hid ${total} values this session (${describe(c)}); restored ${k} placeholders inside Edit, Write and NotebookEdit arguments. Scanning ${rules}.` } }) } diff --git a/plugins/redact/hooks/rules.ts b/plugins/redact/hooks/rules.ts index 48ea2ae..1b5961c 100644 --- a/plugins/redact/hooks/rules.ts +++ b/plugins/redact/hooks/rules.ts @@ -7,6 +7,8 @@ export type Rule = { pattern: RegExp group?: number verify?: (value: string) => boolean + // sees the text and the match start, for context a regex without lookbehind cannot check + before?: (text: string, at: number) => boolean source: string } @@ -141,21 +143,59 @@ function notFiller(v: string): boolean { return entropy(v) >= 2.5 && !/x{6,}|X{6,}|0{8,}|EXAMPLE/.test(v) } +// template and build-time slots: {{secret}}, , __API_KEY__, %s, and a bare `Name=` cut off before its value +function placeholderShaped(v: string): boolean { + return /^(?:\{\{|<|__|%)|(?:\}\}|>|__)$|^[A-Za-z_]+=$/.test(v) +} + function quotedSecret(v: string): boolean { // lowercase words joined by - _ . with no digit read as enum values ('same-origin', 'github_webhook') - return plausibleSecret(v) && entropy(v) >= 2.0 && !/^[a-z]+(?:[-_.][a-z]*)+$/.test(v) + return plausibleSecret(v) && !placeholderShaped(v) && entropy(v) >= 2.0 && !/^[a-z]+(?:[-_.][a-z]*)+$/.test(v) +} + +function codeIdentifier(v: string): boolean { + const digits = v.replace(/\D/g, '').length + const upper = v.replace(/[^A-Z]/g, '').length + const letters = v.replace(/[^A-Za-z]/g, '').length + // word humps (Uint8Array, ToStringUtf8) keep capitals sparse; random strings are about half capitals + return /^[A-Za-z][A-Za-z0-9]*$/.test(v) && digits <= 2 && upper > 0 && upper * 3 <= letters +} + +function codeExpression(v: string): boolean { + if (v.includes('?.') || codeIdentifier(v)) return true + // a member chain (jose.base64url.decode, result.Payload.Data.ToStringUtf8) whose parts read as names, not + // a dotted token whose parts are digit-heavy + const parts = v.split('.') + return parts.length > 1 && parts.every((p) => /^[A-Za-z_$][\w$]*$/.test(p) && p.replace(/\D/g, '').length <= 2) } function bareSecret(v: string): boolean { - if (!plausibleSecret(v) || entropy(v) < 3.0) return false + if (!plausibleSecret(v) || placeholderShaped(v) || entropy(v) < 3.0) return false // a code identifier or member chain with no digit (getToken, process.env.KEY, my-secret-name) is not a value if (!/\d/.test(v) && /^[A-Za-z_$][\w$]*(?:[.\-][A-Za-z_$][\w$]*)*$/.test(v)) return false - return true + return !codeExpression(v) +} + +function keywordStart(text: string, at: number): boolean { + if (at === 0) return true + const prev = text.charAt(at - 1) + if (prev === ':') return false + if (!/[A-Za-z0-9]/.test(prev) || /\\[nrt]$/.test(text.slice(Math.max(0, at - 2), at))) return true + // inside a word only a camelCase hump counts (dbPassword, clientSecret), never the auth of OAuth + return /^[A-Z][a-z]/.test(text.slice(at, at + 2)) && !(/o/i.test(prev) && /^auth/i.test(text.slice(at, at + 4))) } function dbUrlHasRealPassword(url: string): boolean { const password = /^[^:]+:\/\/[^:@\/]*:([^@\/]*)@/.exec(url)?.[1] ?? '' - return password.length > 0 && plausibleSecret(password) && !/^(?:pass|pwd|secret)$/i.test(password) + return ( + password.length > 0 && + plausibleSecret(password) && + !/^(?:pass|pwd|secret|password|%[sd]|\{\{.*\}\}|<.*>)$/i.test(password) + ) +} + +function notTemplated(text: string, at: number): boolean { + return text.slice(Math.max(0, at - 2), at) !== '{{' } const FILE_TLDS = new Set([ @@ -347,15 +387,25 @@ export const RULES: readonly Rule[] = [ label: 'PRIVATE_KEY', kind: 'secret', // the body never holds five dashes, so a BEGIN without an END stops at the next marker instead of scanning 16 KB - pattern: /-----BEGIN (?:RSA |EC |DSA |OPENSSH |ENCRYPTED |PGP )?PRIVATE KEY(?: BLOCK)?-----(?:[^-]|-{1,4}(?!-)){16,16384}?-----END (?:RSA |EC |DSA |OPENSSH |ENCRYPTED |PGP )?PRIVATE KEY(?: BLOCK)?-----/g, + pattern: /-----BEGIN (?:RSA |EC |DSA |OPENSSH |ENCRYPTED |PGP )?PRIVATE KEY(?: BLOCK)?-----(?:[^-]|-{1,4}(?!-)){16,65536}?-----END (?:RSA |EC |DSA |OPENSSH |ENCRYPTED |PGP )?PRIVATE KEY(?: BLOCK)?-----/g, source: GITLEAKS, }, + { + id: 'private-key-truncated', + label: 'PRIVATE_KEY', + kind: 'secret', + // a key cut off before its END line; separators may be real or JSON-escaped newlines, lines may be indented. + // The base64 class has no newline, backslash or space, so each line can split only one way. + pattern: /-----BEGIN (?:RSA |EC |DSA |OPENSSH |ENCRYPTED |PGP )?PRIVATE KEY(?: BLOCK)?-----[ \t]{0,16}(?:(?:\r?\n|\\r\\n|\\n)[ \t]{0,16}[A-Za-z][A-Za-z0-9-]{0,63}:[ \t][^\r\n\\]{0,256}){0,8}(?:(?:\r?\n|\\r\\n|\\n)[ \t]{0,16}(?=\r?\n|\\r\\n|\\n))?(?:(?:\r?\n|\\r\\n|\\n)[ \t]{0,16}[A-Za-z0-9+\/=]{16,128}){2,2048}(?![A-Za-z0-9+\/=])/g, + source: 'https://www.rfc-editor.org/rfc/rfc7468', + }, { id: 'db-connection-url', label: 'DB_URL', kind: 'secret', pattern: /\b(?:postgres(?:ql)?|mysql|mariadb|mongodb(?:\+srv)?|rediss?|amqps?):\/\/[^\s:@\/'"`<>]{0,128}:[^\s@\/'"`<>]{1,256}@[^\s\/'"`<>?#@(){}\[\],;]{1,256}(?:[\/?#][^\s'"`<>(){}\[\],;]{0,1024})?/gi, verify: dbUrlHasRealPassword, + before: notTemplated, source: GITHUB_PATTERNS, }, { @@ -374,15 +424,18 @@ export const RULES: readonly Rule[] = [ pattern: /(?:api[_-]?key|secret(?:[_-]?key)?|token|password|passwd|pwd|auth|credential)s?["'`]?[ \t]{0,5}(?::=|=>|[=:])[ \t]{0,5}["'`]([^\s"'`]{8,256})["'`]/gi, group: 1, verify: quotedSecret, + before: keywordStart, source: GITLEAKS, }, { id: 'generic-secret', label: 'SECRET', kind: 'secret', - pattern: /(?:api[_-]?key|secret(?:[_-]?key)?|token|password|passwd|pwd|auth|credential)s?["'`]?[ \t]{0,5}(?::=|=>|[=:])[ \t]{0,5}([^\s"'`;,(){}<>\[\]]{8,256})(?![^\s"'`;,(){}<>\[\]])/gi, + // a literal \n, \r or \t (backslash and letter) ends the value, as in a dotenv string inside source code + pattern: /(?:api[_-]?key|secret(?:[_-]?key)?|token|password|passwd|pwd|auth|credential)s?["'`]?[ \t]{0,5}(?::=|=>|[=:])[ \t]{0,5}((?:[^\s"'`;,(){}<>\[\]\\]|\\(?![nrt])){8,256})(?!(?!\\[nrt])[^\s"'`;,(){}<>\[\]])/gi, group: 1, verify: bareSecret, + before: keywordStart, source: GITLEAKS, }, { @@ -468,6 +521,98 @@ function compile(pattern: RegExp): Compiled { type Ranked = Hit & { order: number } +// Rules anchored on a fixed prefix or marker. Only these run on decoded base64: the generic and PII rules +// would fire on arbitrary decoded prose. +const WRAPPABLE = new Set([ + 'aws-access-key', 'github-token', 'gitlab-pat', 'slack-token', 'slack-webhook', 'stripe-live-key', 'sendgrid-key', + 'twilio-key', 'google-api-key', 'google-oauth-secret', 'anthropic-key', 'openai-key', 'huggingface-token', + 'npm-token', 'pypi-token', 'private-key', 'private-key-truncated', 'db-connection-url', 'jwt', +]) +const B64_MIN = 24 +const B64_MAX = 8192 +const B64_RUNS = 200 + +const B64_VALUE = new Int8Array(128).fill(-1) +for (let i = 0; i < 64; i++) B64_VALUE[B64URL.charCodeAt(i)] = i +B64_VALUE[43] = 62 +B64_VALUE[47] = 63 + +function b64Value(code: number): number { + return code < 128 ? B64_VALUE[code]! : -1 +} + +// Maximal runs of the base64 alphabet (standard or url-safe, never mixed) with optional padding. +// Padded or standard runs must be a multiple of 4 long; unpadded url-safe runs only need a valid tail. +function base64Runs(text: string): [number, number][] { + const runs: [number, number][] = [] + let i = 0 + while (i < text.length && runs.length < B64_RUNS) { + if (b64Value(text.charCodeAt(i)) < 0) { + i++ + continue + } + let j = i + let std = false + let url = false + for (; j < text.length; j++) { + const c = text.charCodeAt(j) + if (b64Value(c) < 0) break + if (c === 43 || c === 47) std = true + else if (c === 45 || c === 95) url = true + } + let k = j + while (k < text.length && k - j < 2 && text.charCodeAt(k) === 61) k++ + const body = j - i + const total = k - i + const shaped = k > j ? total % 4 === 0 : body % 4 === 0 || (!std && body % 4 !== 1) + if (total >= B64_MIN && total <= B64_MAX && !(std && url) && shaped) runs.push([i, k]) + i = k + } + return runs +} + +// Decodes text[start, end) to a byte string, or null once more than a tenth of the bytes are not printable ASCII. +function decodePrintable(text: string, start: number, end: number): string | null { + let stop = end + while (stop > start && text.charCodeAt(stop - 1) === 61) stop-- + const maxBad = Math.floor(((stop - start) * 3) / 40) + let bad = 0 + let acc = 0 + let bits = 0 + let out = '' + for (let i = start; i < stop; i++) { + acc = ((acc << 6) | b64Value(text.charCodeAt(i))) & 0xffffff + bits += 6 + if (bits >= 8) { + bits -= 8 + const byte = (acc >> bits) & 0xff + if (!(byte >= 0x20 && byte <= 0x7e) && byte !== 0x09 && byte !== 0x0a && byte !== 0x0d && ++bad > maxBad) return null + out += String.fromCharCode(byte) + } + } + return out +} + +function firstWrappedRule(decoded: string, off: ReadonlySet | undefined): number { + for (let order = 0; order < RULES.length; order++) { + const rule = RULES[order]! + if (!WRAPPABLE.has(rule.id) || off?.has(rule.id) || off?.has(`${rule.id}-base64`)) continue + const { re } = compile(rule.pattern) + const g = rule.group ?? 0 + re.lastIndex = 0 + for (let m = re.exec(decoded); m !== null; m = re.exec(decoded)) { + const raw = m[g] + if (raw && (!rule.before || rule.before(decoded, m.index)) && (!rule.verify || rule.verify(raw))) { + re.lastIndex = 0 + return order + } + re.lastIndex = m.index + 1 + } + re.lastIndex = 0 + } + return -1 +} + export function scan(text: string, opts: { pii?: boolean; off?: ReadonlySet } = {}): Hit[] { const spans: [number, number][] = [] PLACEHOLDER.lastIndex = 0 @@ -505,7 +650,7 @@ export function scan(text: string, opts: { pii?: boolean; off?: ReadonlySet start < e && end > s) - if (inPlaceholder || (rule.verify && !rule.verify(value))) { + if (inPlaceholder || (rule.before && !rule.before(masked, m.index)) || (rule.verify && !rule.verify(value))) { re.lastIndex = m.index + 1 continue } @@ -514,6 +659,16 @@ export function scan(text: string, opts: { pii?: boolean; off?: ReadonlySet a.start - b.start || b.end - b.start - (a.end - a.start) || a.order - b.order) const hits: Hit[] = [] let lastEnd = -1 diff --git a/plugins/redact/tests/register.test.ts b/plugins/redact/tests/register.test.ts index d29c4c6..b5a791b 100644 --- a/plugins/redact/tests/register.test.ts +++ b/plugins/redact/tests/register.test.ts @@ -23,12 +23,12 @@ describe('register', () => { test('/redact says nothing was hidden in a fresh session, secrets only', async ($, on) => { on('ui.log', () => ({ value: undefined })) - expect((await $.command.run(run)).text).toBe('redact: nothing hidden yet this session. Scanning secrets only (set pii to true for email, phone, cards, national ids).') + expect((await $.command.run(run)).text).toBe('nothing hidden yet this session. Scanning secrets only (set pii to true for email, phone, cards, national ids).') }) test('/redact names PII scanning when the option is on', { options: { pii: true } }, async ($, on) => { on('ui.log', () => ({ value: undefined })) - expect((await $.command.run(run)).text).toBe('redact: nothing hidden yet this session. Scanning secrets and PII.') + expect((await $.command.run(run)).text).toBe('nothing hidden yet this session. Scanning secrets and PII.') }) test('Edit, Write and NotebookEdit pass through unchanged when no placeholder is known', async ($, on) => { @@ -40,7 +40,7 @@ describe('register', () => { await $.tool.call({ tool: 'Write', file_path: '/work/x', content: `C=${UNKNOWN}` }) await $.tool.call({ tool: 'NotebookEdit', notebook_path: '/work/n.ipynb', new_source: `D=${UNKNOWN}` }) expect(seen).toEqual([`A=${UNKNOWN}`, `B=${UNKNOWN}`, `C=${UNKNOWN}`, `D=${UNKNOWN}`]) - expect((await $.command.run(run)).text).toMatch(/^redact: nothing hidden yet/) + expect((await $.command.run(run)).text).toMatch(/^nothing hidden yet/) }) test('Bash is never touched', async ($, on) => { @@ -87,7 +87,7 @@ describe('prompt.submit', () => { const { sent, lines } = submitStub(on) await $.prompt.submit({ text: `my key is ${KEY} keep it`, origin: { kind: 'composer' }, wait: false }) expect(sent).toEqual([expect.stringMatching(/^my key is \[REDACTED:AWS_KEY#[0-9a-f]{8}\] keep it$/)]) - expect(lines).toEqual(['redact: hid 1 AWS_KEY in your prompt']) + expect(lines).toEqual(['hid 1 AWS_KEY in your prompt']) }) test('a clean prompt passes through as the same text with no line', async ($, on) => { @@ -111,7 +111,7 @@ describe('prompt.submit', () => { expect(edits).toEqual([`SECRET=${KEY}`, `# moved\nSECRET=${KEY}`]) expect(shells).toEqual([`echo ${p}`]) expect((await $.command.run({ command: 'redact', args: '', origin: { kind: 'composer' }, presentation: { isFullscreen: false, columns: 80 } })).text) - .toBe('redact: hid 1 values this session (1 AWS_KEY); restored 2 placeholders inside Edit, Write and NotebookEdit arguments. Scanning secrets only (set pii to true for email, phone, cards, national ids).') + .toBe('hid 1 values this session (1 AWS_KEY); restored 2 placeholders inside Edit, Write and NotebookEdit arguments. Scanning secrets only (set pii to true for email, phone, cards, national ids).') }) test('with pii on, an email in the prompt is hidden; off by default', { options: { pii: true } }, async ($, on) => { diff --git a/plugins/redact/tests/rules.test.ts b/plugins/redact/tests/rules.test.ts index 11efc1f..7143b31 100644 --- a/plugins/redact/tests/rules.test.ts +++ b/plugins/redact/tests/rules.test.ts @@ -104,6 +104,45 @@ const PEMS = [ pem('EC ', 'MHcCAQEEIFakeFakeFakeKeyMaterialForTestsOnly0123456789oAoGCCqGSM49'), cat('-----BEGIN PGP ', 'PRIVATE KEY BLOCK-----\n\nlQOYBGFakeTestKeyBlockNotRealAtAll0123456789abcdef\n=AbCd\n-----END PGP ', 'PRIVATE KEY BLOCK-----'), ] as const +// random bytes, base64 encoded: shaped like key body lines, decode to nothing +const BODY = [ + 'Cwt6LdVeAeNtMwcg7fRuJeOEthYUxl+wh4mkBtvO3jeUrw6g/pHKeeCB9SvYEYgX', + '2XRFjCcxb2tTNclcj5+Awuv/ipA28sjg3IidDsJkO4EZ0M8p3rO6SkO/yxpz79Mp', + 'HyXFxEpnRzuEyKVCHLmrNOd9GUjm8me/zYZ+RHB5YE16DafnE9UIkFndQyUYyD9A', + 'viGE2jtGDOqocVvXf7VvHLl3tpKIB+p8hf6CiGv/uvUrdlJEeRoSuFctewnhJXtu', + 'Ic518be1I790YaKVaf0gfqFex/mBykHUdfbQSUzN+evjihRwMABG2U0zdsKdAIh1', + 'k4iArmiIAdr6UnWhHa2gH/vJyXDmdiDdZQZrZopClQ+Pg7e60BPGaz1dKx9Aabs8', + 'sjr/Cjgt0RnL0t6cEeSt8IfEDlbL8ciem3KvIJyTHO8UhYdDwJKUg2hxgJgBVfaj', + '9nFanOPHPV8QKVqDlK/d+WKf46gmTUadKFa3B3EkndU9u1wL9D8V6RUkRlhsWF2b', + 'fCmL0O+VBHaA/x2aujB23ZHPitel+1fD8U8Ymf6KW3EMm3hPYohQizodYPiKwxBR', + 'T7x0W6hEG71ToPfYF+xcpwKxbc9zr0kya9rgeneHnojD5P89TH1HjyXLSz84iSgD', +] as const +// a body past the old 16384 character cap, as in a large PGP export +const LONG_BODY = Array.from({ length: 300 }, (_, i) => BODY[i % BODY.length]).join('\n') +const LONG_PGP = cat('-----BEGIN PGP ', 'PRIVATE KEY BLOCK-----\n\n', LONG_BODY, '\n=AbCd\n-----END PGP ', 'PRIVATE KEY BLOCK-----') +const head = (kind: string, sep: string, ...lines: string[]): string => + cat('-----BEGIN ', kind, 'PRIVATE KEY-----', ...lines.map((l) => sep + l)) +const TRUNCATED = [ + head('RSA ', '\n', BODY[0], BODY[1], BODY[2]), + head('RSA ', '\n', 'Proc-Type: 4,ENCRYPTED', 'DEK-Info: AES-256-CBC,0F9D4DAE9EC28C165D631F076B265F06', '', BODY[3], BODY[4]), + head('', '\\n', BODY[5], BODY[6], BODY[7]), + head('OPENSSH ', '\r\n', BODY[8], BODY[9]), + head('EC ', '\n ', BODY[1], BODY[3]), + head('PGP ', '\n', 'Version: GnuPG v2', '', BODY[2], BODY[4]).replace('KEY-----', 'KEY BLOCK-----'), +] as const + +const B64_ALPHABET = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/' +// what `base64` or `base64 -w0` prints; url gives the unpadded url-safe form +const b64 = (s: string, url = false): string => { + let out = '' + for (let i = 0; i < s.length; i += 3) { + const n = (s.charCodeAt(i) << 16) | ((i + 1 < s.length ? s.charCodeAt(i + 1) : 0) << 8) | (i + 2 < s.length ? s.charCodeAt(i + 2) : 0) + const quad = [18, 12, 6, 0].map((shift) => B64_ALPHABET.charAt((n >> shift) & 63)) + const keep = i + 2 < s.length ? 4 : i + 1 < s.length ? 3 : 2 + out += quad.slice(0, keep).join('') + (url ? '' : '='.repeat(4 - keep)) + } + return url ? out.replace(/\+/g, '-').replace(/\//g, '_') : out +} const DB_URLS = [ cat('postgres', '://admin:S3cr3tPa55w0rd', '@db.internal.test:5432/app'), cat('mongodb+srv', '://svc_user:Xk9mP2vL7qR4', '@cluster0.abcde.mongodb.net/prod?retryWrites=true'), @@ -391,6 +430,7 @@ const SECRET_CASES: Case[] = [ { text: `"${PEMS[1]}"`, value: PEMS[1] }, { text: `${PEMS[2]}`, value: PEMS[2] }, { text: `gpg export\n${PEMS[3]}\ndone`, value: PEMS[3] }, + { text: `gpg --export-secret-keys --armor\n${LONG_PGP}\n`, value: LONG_PGP }, ], negatives: [ '-----BEGIN PUBLIC KEY-----\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEFakePublicKeyMaterial\n-----END PUBLIC KEY-----', @@ -398,6 +438,25 @@ const SECRET_CASES: Case[] = [ cat('-----BEGIN RSA ', 'PRIVATE KEY----- truncated, no end marker'), ], }, + { + id: 'private-key-truncated', + label: 'PRIVATE_KEY', + positives: [ + { text: `$ head -4 id_rsa\n${TRUNCATED[0]}\n$ `, value: TRUNCATED[0] }, + { text: `${TRUNCATED[1]}\n... (40 more lines)`, value: TRUNCATED[1] }, + { text: `{"type": "service_account", "private_key": "${TRUNCATED[2]}`, value: TRUNCATED[2] }, + { text: `${TRUNCATED[3]}\r\n`, value: TRUNCATED[3] }, + { text: `tls:\n key: |\n ${TRUNCATED[4]}\n cert: pending`, value: TRUNCATED[4] }, + { text: `${TRUNCATED[5]}`, value: TRUNCATED[5] }, + ], + negatives: [ + cat('-----BEGIN RSA ', 'PRIVATE KEY-----\n', BODY[0], '\n(output truncated)'), + cat('-----BEGIN PUBLIC KEY-----\n', BODY[1], '\n', BODY[2]), + cat('-----BEGIN CERTIFICATE-----\n', BODY[3], '\n', BODY[4]), + cat('-----BEGIN ', 'PRIVATE KEY-----\npaste the rest of the key here\nand keep the END line'), + cat('-----BEGIN ', 'PRIVATE KEY-----\nMIIE\nshort\nlines'), + ], + }, { id: 'db-connection-url', label: 'DB_URL', @@ -412,6 +471,12 @@ const SECRET_CASES: Case[] = [ 'postgres://localhost:5432/app', 'mongodb://${DB_USER}:${DB_PASS}@mongo.test/db', 'mysql://root:@127.0.0.1:3306/app', + 'return sql.Open("postgres", fmt.Sprintf("postgres://postgres:%s@db:5432/example?sslmode=disable", string(bin)))', + 'dsn := fmt.Sprintf("mysql://app:%d@db.test:3306/app", port)', + '`csvsql --insert --db "{{mysql://benutzer:passwort@host/datenbank}}" {{pfad/zu/datei.csv}}`', + '`csvsql --insert --db "{{mysql://pengguna:kata_sandi@host/basis_data}}" {{jalan/menuju/data.csv}}`', + 'DATABASE_URL=postgres://app:{{db_password}}@db.test/app', + 'DATABASE_URL=mysql://root:PASSWORD@db.test/app', ], }, { @@ -437,6 +502,7 @@ const SECRET_CASES: Case[] = [ { text: `const apiKey = '${cat('OWzE65iZIr', 'tvsKC8rnto')}'`, value: cat('OWzE65iZIr', 'tvsKC8rnto') }, { text: `{"client_secret": "Zx8#kL2!pQ9@vR4$"}`, value: 'Zx8#kL2!pQ9@vR4$' }, { text: `DB_PASSWORD="s0me-Pa55word"`, value: 's0me-Pa55word' }, + { text: `const clientSecret = "Zx8#kL2!pQ9@vR4$"`, value: 'Zx8#kL2!pQ9@vR4$' }, ], negatives: [ 'password: "password"', @@ -445,6 +511,10 @@ const SECRET_CASES: Case[] = [ 'secret = "changeme"', '"token": "xxxxxxxxxxxxxxxx"', 'password: "%DB_PASSWORD%"', + " apiKey: '__API_KEY__',", + 'password: "%(db_password)s"', + '"api_key": "{{ secrets.API_KEY }}"', + "token: ''", ], }, { @@ -455,6 +525,9 @@ const SECRET_CASES: Case[] = [ { text: `db_password: Tr0ub4dor&3xyz\n`, value: 'Tr0ub4dor&3xyz' }, { text: `run --token=${cat('1e64c201a6ccba54', '93ddc9c2cb8a61df')} --verbose`, value: cat('1e64c201a6ccba54', '93ddc9c2cb8a61df') }, { text: `export SECRET_KEY=k3J9mQ2xV7pL4nR8`, value: 'k3J9mQ2xV7pL4nR8' }, + { text: `dbPassword: Xk9mPqLvRtZw7`, value: 'Xk9mPqLvRtZw7' }, + { text: `SECRET_KEY=${cat('Zm9vYmFyYmF6cXV4MTIz', 'NDU2Nzg5MA==')}`, value: cat('Zm9vYmFyYmF6cXV4MTIz', 'NDU2Nzg5MA==') }, + { text: String.raw`dotenv.parse('SERVER=localhost\nPASSWORD=Tr0ub4dor&3xyz\nDB=tests\n')`, value: 'Tr0ub4dor&3xyz' }, ], negatives: [ 'const token = getToken()', @@ -465,6 +538,18 @@ const SECRET_CASES: Case[] = [ 'pwd = /usr/local/bin', 'password=aaaaaaaaaaaa', 'token: 1727950000', + "// 'urn:ietf:params:oauth:jwk-thumbprint:sha-256:NzbLsXh8uDCcd-6MNwXF4W_7noWXFZAfHkxZsRGC9Xs'", + 'const uri = `urn:ietf:params:OAuth:jwk-thumbprint:sha-256:${thumbprint}`', + '`sf force:auth:web:login --setalias {{organization}} --instanceurl {{organization_url}}`', + "`qm remote-migrate {{vmid}} {{target_vmid}} 'apitoken=PVEAPIToken={{user}}@{{realm}}!{{token}}={{secret}},host={{address}}' --target-bridge {{bridge}}`", + ' auth: options?.auth,', + " string alloyDBPassword = result.Payload.Data.ToStringUtf8().TrimEnd('\\r', '\\n');", + " * const secret = jose.base64url.decode('Qm9sZEZha2VWYWx1ZUZvclRlc3RzT25seTEyMzQ')", + 'declare const secret: Uint8Array', + ' secret: Uint8Array', + String.raw` 'WINDIR="C:\\\\Users\\\\me\\\\"\nAPI_KEY=secret\nPORT=3000\n',`, + String.raw`const RPayload = dotenv.parse(Buffer.from('SERVER=localhost\rPASSWORD=password\rDB=tests\r'))`, + String.raw`const RNPayload = dotenv.parse(Buffer.from('SERVER=localhost\r\nPASSWORD=password\r\nDB=tests\r\n'))`, ], }, ] @@ -561,6 +646,38 @@ const PII_CASES: Case[] = [ const CASES = [...SECRET_CASES, ...PII_CASES] +type Wrapped = { text: string; value: string; id: string; label: string } +const ENV_FILE = cat('AWS_REGION=us-east-1\nGITHUB_TOKEN=', GHP, '\nLOG_LEVEL=debug\n') +const SLACK_JSON = JSON.stringify({ api: 'https://api.test', token: SLACK[0] }) +const WRAPPED: Wrapped[] = [ + { text: `echo ${b64(AKIA)} | base64 -d`, value: b64(AKIA), id: 'aws-access-key-base64', label: 'AWS_KEY' }, + { text: `ENV_FILE=${b64(ENV_FILE)}`, value: b64(ENV_FILE), id: 'github-token-base64', label: 'GITHUB_TOKEN' }, + { text: `{"config": "${b64(SLACK_JSON)}"}`, value: b64(SLACK_JSON), id: 'slack-token-base64', label: 'SLACK_TOKEN' }, + { text: `data:\n tls.key: ${b64(PEMS[0])}\n`, value: b64(PEMS[0]), id: 'private-key-base64', label: 'PRIVATE_KEY' }, + { text: `?state=${b64(cat('db=', DB_URLS[0]), true)}&x=1`, value: b64(cat('db=', DB_URLS[0]), true), id: 'db-connection-url-base64', label: 'DB_URL' }, + { text: `{"auths":{"registry.test":{"auth":"${b64(cat('ci:', NPM[0]))}"}}}`, value: b64(cat('ci:', NPM[0])), id: 'npm-token-base64', label: 'NPM_TOKEN' }, +] +const WRAPPED_NEGATIVES = [ + '', + '"integrity": "sha512-5M53BsOqcReBM10CEzSaGC4lddbxUMy9v+iLQ8GPQl2d27nb8yuN8GHr8JhtKZe3H3AIIBvsdZDfCqO8lqGa6Q=="', + '"integrity": "sha1-kBRmpbtS7m3mPj44UoXmkNXLpCQ="', + 'blob: 6iji9g/bOXS4N9cXTAzcF1F5RhZEq7jDCEsKwJqtlYmyUJfjPNZYVP9wodu0PZLEAAbozl5ElVVTkFjFdxW+9nkr4wy2w4j/L+Giz4ML+Qz5vmSN/5TSr4o9cC7cec3E', + 'nonce=CX3Idgv6Qwremr1p7Ii0x5Uq4oPUfxTYXLE0qwH_qd_7', + `msg: ${b64('build 4521 passed on the main branch, nothing to see here')}`, + `doc: ${b64(cat('aws_access_key_id = AKIA', 'IOSFODNN7EXAMPLE'))}`, + `cfg: ${b64('password=hunter2-Correct! and api_key=k3J9mQ2xV7pL4nR8')}`, + `${b64(AKIA).slice(0, 20)} is too short to decode`, +] + +function seeded(seed: number): () => number { + return (): number => { + seed = (seed + 0x6d2b79f5) | 0 + let t = Math.imul(seed ^ (seed >>> 15), 1 | seed) + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t + return ((t ^ (t >>> 14)) >>> 0) / 4294967296 + } +} + function checkCase(c: Case): void { for (const p of c.positives) { const hits = scan(p.text, { pii: true }) @@ -594,6 +711,18 @@ describe('rules', () => { checkCase(c) }) } + + test(`base64-wrapped: ${WRAPPED.length} positives, ${WRAPPED_NEGATIVES.length} negatives`, ($, on) => { + for (const w of WRAPPED) { + const hits = scan(w.text, { pii: true }) + expect(hits, `one ${w.id} hit for: ${w.text}`).toEqual([ + { id: w.id, label: w.label, kind: 'secret', start: w.text.indexOf(w.value), end: w.text.indexOf(w.value) + w.value.length, value: w.value }, + ]) + } + for (const n of WRAPPED_NEGATIVES) { + expect(scan(n, { pii: true }), `base64 should not match: ${n}`).toEqual([]) + } + }) }) describe('scan', () => { @@ -610,6 +739,10 @@ describe('scan', () => { expect(scan(text).map((h) => h.id)).toEqual(['github-token', 'aws-access-key']) expect(scan(text, { off: new Set(['github-token']) }).map((h) => h.id)).toEqual(['aws-access-key']) expect(scan('mail jane.doe@example.com', { pii: true, off: new Set(['email']) })).toEqual([]) + const wrapped = `raw ${AKIA} wrapped ${b64(AKIA)}` + expect(scan(wrapped).map((h) => h.id)).toEqual(['aws-access-key', 'aws-access-key-base64']) + expect(scan(wrapped, { off: new Set(['aws-access-key-base64']) }).map((h) => h.id)).toEqual(['aws-access-key']) + expect(scan(wrapped, { off: new Set(['aws-access-key']) })).toEqual([]) }) test('an existing placeholder is never matched again', ($, on) => { @@ -633,21 +766,54 @@ describe('scan', () => { expect(sorted.map((h) => h.id)).toEqual(['github-token', 'aws-access-key']) }) - test('2 MB of random base64 scans in under 1000 ms', ($, on) => { - const alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/' - let seed = 0x2f6b1d3a - const next = (): number => { - seed = (seed + 0x6d2b79f5) | 0 - let t = Math.imul(seed ^ (seed >>> 15), 1 | seed) - t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t - return ((t ^ (t >>> 14)) >>> 0) / 4294967296 + test('a whole PEM block wins over its truncated reading', ($, on) => { + for (const block of [PEMS[0], PEMS[1], LONG_PGP, cat(TRUNCATED[1], '\n-----END RSA ', 'PRIVATE KEY-----')]) { + const hits = scan(`key:\n${block}\n`) + expect(hits.map((h) => h.id)).toEqual(['private-key']) + expect(hits[0]?.value).toBe(block) } + }) + + test('a JWT stays a JWT, never base64-wrapped', ($, on) => { + const claims = b64(JSON.stringify({ sub: 'ci', gh: GHP }), true) + const token = cat(b64('{"alg":"HS256","typ":"JWT"}', true), '.', claims, '.', 'c2lnbmF0dXJlLW5vdC1yZWFsLTAxMjM0NQ') + for (const jwt of [...JWTS, token]) { + expect(scan(`Bearer ${jwt}`).map((h) => [h.id, h.value])).toEqual([['jwt', jwt]]) + } + }) + + test('2 MB of random base64 scans in under 1000 ms', ($, on) => { + const next = seeded(0x2f6b1d3a) const chunks: string[] = [] - for (let i = 0; i < 2 * 1024 * 1024; i++) chunks.push(alphabet.charAt(Math.floor(next() * 64))) + for (let i = 0; i < 2 * 1024 * 1024; i++) chunks.push(B64_ALPHABET.charAt(Math.floor(next() * 64))) const blob = chunks.join('') const t0 = performance.now() scan(blob, { pii: true }) const ms = performance.now() - t0 expect(ms).toBeLessThan(1000) }) + + test('2 MB of near-miss keys and decodable base64 scans in under 1000 ms', ($, on) => { + const next = seeded(0x51ed27a9) + const line = (n: number): string => Array.from({ length: n }, () => B64_ALPHABET.charAt(Math.floor(next() * 64))).join('') + const keys: string[] = [] + let size = 0 + while (size < 400 * 1024) { + const part = cat('-----BEGIN ', 'PRIVATE KEY-----\n', Array.from({ length: 1100 }, () => line(64)).join('\n'), '\n', line(200), '\n') + keys.push(part) + size += part.length + } + const prose = Array.from({ length: 6000 }, (_, i) => String.fromCharCode(97 + (i % 26)) + (i % 7 === 0 ? ' ' : '')).join('').slice(0, 6000) + const runs: string[] = [] + while (size < 2 * 1024 * 1024) { + const run = b64(prose) + runs.push(run) + size += run.length + 1 + } + const text = keys.join('') + runs.join(' ') + const t0 = performance.now() + scan(text, { pii: true }) + const ms = performance.now() - t0 + expect(ms).toBeLessThan(1000) + }) }) diff --git a/site/index.html b/site/index.html index 884c753..881fcce 100644 --- a/site/index.html +++ b/site/index.html @@ -10,21 +10,21 @@ --bg: #FAF9F5; --bg-alt: #F0EEE6; --text: #141413; - --muted: #5E5D59; + --muted: #3D3D3A; --faint: #87867F; --accent: #D97757; - --accent-hover: #CC785C; + --accent-hover: #C6613F; --kraft: #D4A27F; --manilla: #EBDBBC; --panel: #F0EEE6; --panel-warm: #F5ECDA; - --line: #E3DFD3; + --line: #E8E6DC; --code-bg: #141413; --code-text: #F0EEE6; --code-dim: #87867F; --on-accent: #FAF9F5; - --serif: "Iowan Old Style", "Palatino Linotype", Palatino, Georgia, "Times New Roman", serif; - --sans: system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; + --serif: "Anthropic Serif", "Tiempos Text", "Iowan Old Style", "Palatino Linotype", Palatino, Georgia, "Times New Roman", serif; + --sans: "Styrene B", "Styrene A", system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; --mono: ui-monospace, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; color-scheme: light; } @@ -209,7 +209,7 @@

Nothing leaves the machine

Edits get the real value

Placeholders are swapped back only inside Edit, Write and NotebookEdit arguments, so an edit to a line that held a key still matches the file. Bash never gets the real value.

-

Kit tests cover the argument rewrite. A live edit in a logged-in session has not been run yet.

+

Driven live on 2.1.288: a key typed in the prompt, Claude asked to save it, the file on disk holds the real key while every conversation row holds the placeholder.

The transcript is clean too

@@ -222,10 +222,10 @@

The transcript is clean too

What it catches

-

30 rules. Each one has a fake example and a public source in RULES.md. Any rule can be switched off by id.

+

31 rules. Each one has a fake example and a public source in RULES.md. Any rule can be switched off by id.

-

Secrets always on, 23 rules

+

Secrets always on, 24 rules

  • AWS access key ids and secret access keys
  • GitHub, GitLab, npm, PyPI and Hugging Face tokens
    • @@ -235,10 +235,11 @@

    Secrets always on, 23 rules

  • Anthropic and OpenAI keys
  • Azure storage account keys
  • JWTs (header must decode to JSON with alg)
    • -
  • PEM private key blocks
    • +
  • PEM private key blocks, complete or truncated
  • Database URLs that carry a password
  • Authorization header values
  • Generic password=, api_key= style values with enough entropy
    • +
  • Any prefix-shaped token above wrapped in base64
@@ -262,7 +263,8 @@

PII opt-in, 7 rules

What it does not do