From c0ca88d4c8eecc308f03f56c7495e036d7ed4410 Mon Sep 17 00:00:00 2001 From: Pheidon Date: Sun, 26 Jul 2026 15:30:44 +0000 Subject: [PATCH] feat: harden public provenance boundaries Restrict public metadata to an explicit schema allowlist, reject unknown and credential-shaped identity fields, validate exact redaction evidence, and document the confirmed provenance threat model. Emit and validate strict version-2 manifests, retain version 1 behind an explicit hardened historical reader, and record the remote-sink hard stops required by issue #61. Signed-off-by: Pheidon --- docs/bootstrap/public-provenance.md | 8 +- .../ADR-0006-public-provenance-v2.md | 53 +++++ docs/security/bootstrap-threat-model.md | 141 +++++++++++++ src/provenance.ts | 192 +++++++++++++++--- tests/provenance.test.ts | 151 +++++++++++++- 5 files changed, 509 insertions(+), 36 deletions(-) create mode 100644 docs/decisions/ADR-0006-public-provenance-v2.md create mode 100644 docs/security/bootstrap-threat-model.md diff --git a/docs/bootstrap/public-provenance.md b/docs/bootstrap/public-provenance.md index 6bb95e3..d845fd8 100644 --- a/docs/bootstrap/public-provenance.md +++ b/docs/bootstrap/public-provenance.md @@ -1,7 +1,11 @@ # Public Provenance -`bootstrap provenance create --input provenance-input.json --output provenance/runs/.json` redacts metadata and writes the versioned public manifest. `bootstrap provenance validate --input provenance/runs/.json` validates it before publication. +`bootstrap provenance create --input provenance-input.json --output provenance/runs/.json` redacts metadata and writes a version-2 public manifest. `bootstrap provenance validate --input provenance/runs/.json` validates it before publication. -The public manifest records the repository, immutable commit SHA, workflow run, reviewer lineage, and safe metadata. It rejects credential-like literals; generators replace detected values with `[REDACTED:CREDENTIAL]` and record the replacement count. +The public manifest records the repository, immutable commit SHA, workflow run, reviewer lineage, and explicitly allowlisted metadata. The only metadata keys accepted are `policy`, `generator`, `aiProvider`, `aiModel`, `promptHash`, and `changeClass`; unknown keys and unknown fields at any schema boundary fail validation rather than being copied or silently removed. + +Credential-like literals in allowlisted metadata become typed `[REDACTED:CREDENTIAL]` placeholders. Validation rejects credential-like values in identity fields and rejects redaction counts that do not exactly match the emitted placeholders. The detector is defense in depth, not permission to pass arbitrary logs, prompts, tool output, customer data, or other private material into the public generator. + +The default validator accepts only version 2 so publication and merge gates cannot downgrade to the permissive legacy metadata shape. Library consumers may use the explicit `readLegacyPublicProvenance` compatibility reader for historical version-1 artifacts; it rejects unknown fields and credential-like literals across recognized public fields. Version 1 still permits arbitrary metadata keys and does not satisfy the current allowlist contract. Migrate by supplying the intended allowlisted fields to `provenance create` and publishing the resulting version-2 manifest rather than relabeling an existing artifact. This public contract deliberately does not include a private provenance sink, encryption material, bucket identity, or long-lived credentials. Those remain a separately reviewed issue #61 capability using short-lived runtime identity. diff --git a/docs/decisions/ADR-0006-public-provenance-v2.md b/docs/decisions/ADR-0006-public-provenance-v2.md new file mode 100644 index 0000000..cd8d175 --- /dev/null +++ b/docs/decisions/ADR-0006-public-provenance-v2.md @@ -0,0 +1,53 @@ +# ADR-0006: Make public provenance version 2 fail closed + +Status: Accepted + +Date: 2026-07-23 + +Decision owners: Bootstrap maintainers + +Material notification: [Bootstrap issue #61](https://github.com/OMT-Global/bootstrap/issues/61) and the implementing pull request + +## Context + +Public provenance version 1 accepts arbitrary metadata keys and relies on credential-pattern detection. That is useful as an initial redaction foundation, but it cannot establish that public output contains only intentionally public fields. The local public manifest is also distinct from the existing cosign-signed AI attestation workflow, and private encryption, remote sinks, audited reads, retention, and fail-closed material gates are not implemented. + +The confirmed security boundary permits only schema-allowlisted metadata in public output. Private model and tool evidence may be sensitive, literal credentials are prohibited from every output, remote storage must use short-lived OIDC with separate read and write roles, and production sink use requires independent security review plus explicit human approval. + +## Decision + +1. New public provenance manifests use schema version 2. +2. Version 2 uses strict object boundaries and permits only `policy`, `generator`, `aiProvider`, `aiModel`, `promptHash`, and `changeClass` metadata keys. +3. Default creation and validation are version-2-only. Historical version-1 artifacts are available through an explicit strict compatibility reader and cannot satisfy a current publication or merge gate. +4. Credential-shaped identity values fail validation. Allowlisted metadata is redacted to a typed placeholder, and the recorded replacement count must match the emitted placeholders exactly. +5. The output bound accounts for worst-case redaction expansion while input remains bounded. +6. This decision does not authorize a private sink, choose a storage provider, expose private data, approve retention exceptions, or claim that public schema output is signed. + +## Consequences + +- Consumers constructing current manifests must use the exported current schema version and migrate intended public metadata into the allowlisted fields. +- Safe historical version-1 artifacts remain readable, but callers must opt into that path and cannot pass them to the default publication validator. +- The public schema and the signed AI attestation remain separate until a later issue #61 slice defines canonical serialization, authenticated reviewer lineage, subject binding, and signature verification. +- Remote encryption, storage, retention, read auditing, and merge blocking remain independently reviewable work. + +## Alternatives considered + +### Tighten version 1 in place + +Rejected because previously valid arbitrary metadata would silently change meaning under the same version. + +### Let the default validator accept versions 1 and 2 + +Rejected because a caller could select version 1 to bypass the current allowlist. + +### Remove version-1 support + +Rejected because historical artifacts can remain safely readable through an explicit non-publication path. + +## Security and privacy + +Pattern detection is defense in depth, not a classifier for arbitrary private material. Logs, prompts, tool output, customer data, encryption material, credentials, and physical sink configuration must not be passed to the public generator. The repository threat model in `docs/security/bootstrap-threat-model.md` governs follow-up design and review. + +## Revisit conditions + +Revisit when the signed public format is unified with the local schema, before enabling any private remote sink, or if the allowlisted public fields require a breaking change. diff --git a/docs/security/bootstrap-threat-model.md b/docs/security/bootstrap-threat-model.md new file mode 100644 index 0000000..86a51e7 --- /dev/null +++ b/docs/security/bootstrap-threat-model.md @@ -0,0 +1,141 @@ +# Bootstrap provenance threat model + +## Executive summary + +Bootstrap is a local TypeScript control-plane CLI plus GitHub Actions workflow set. For provenance, the highest risks are publishing sensitive input through a public manifest, accepting forged reviewer or subject lineage, trusting an unsigned or improperly verified manifest, and later allowing private-bundle or sink failures to fail open. The current local public contract reduces disclosure risk with strict schema allowlists and typed redaction, while signed public lineage, encrypted private bundles, remote sinks, audited reads, retention, and material-merge enforcement remain explicit follow-up controls rather than assumed protections. + +## Scope and assumptions + +- In scope: `src/provenance.ts`, provenance commands in `src/cli.ts`, the shared filter use in `src/notifications.ts`, `.github/workflows/ai-attestation-reusable.yml`, its renderer in `src/archetypes.ts`, and their tests and operator documentation. +- Runtime scope is the local CLI and generated files. CI scope is the GitHub-hosted attestation workflow and artifact handoff. These are separate provenance formats today. +- Public provenance is limited to schema-allowlisted metadata. Private model/tool evidence may be sensitive, but literal credentials are prohibited from both public and future private output. +- A future remote sink must use short-lived OIDC identity with separate write and read roles. Production sink use is disabled until independent security review and explicit human approval. +- Out of scope for this slice: selecting a storage provider, physical bucket names, production spending, retention exceptions, private-data exposure approval, and implementation of encryption or audited remote reads. +- Open questions for later slices: the authoritative source for reviewer lineage; the production sink provider and jurisdiction; retention/Object Lock duration; and the exact material-change capture gate. These choices materially affect integrity, confidentiality, and availability rankings. + +## System model + +### Primary components + +- The `bootstrap provenance create` and `validate` commands read operator-selected JSON files, invoke the Zod contract, and write or print public JSON (`src/cli.ts`, `src/provenance.ts`). +- The version-2 public contract validates subject, execution, reviewer, allowlisted metadata, and redaction evidence. Default validation is v2-only; an explicit strict reader retains version 1 for historical readability, but legacy manifests do not satisfy the current allowlist policy. Neither format fetches GitHub reviewer state or signs output (`src/provenance.ts`). +- The reusable AI attestation workflow constructs a separate commit-scoped JSON blob, signs it keylessly with cosign using GitHub OIDC, uploads it, then verifies the downloaded artifact (`.github/workflows/ai-attestation-reusable.yml`). +- The renderer projects attestation callers and workflow configuration into managed repositories (`src/archetypes.ts`, `src/manifest.ts`). +- Material-action notifications reuse the credential detector before sending governed messages to GitHub or configured webhooks (`src/notifications.ts`, `src/github/client.ts`). +- Private encryption, remote sink writes, read auditing, retention enforcement, and merge-blocking capture are not implemented. + +### Data flows and trust boundaries + +- Operator JSON -> Bootstrap CLI: repository, commit, workflow, reviewer, and allowlisted metadata cross a local file boundary. The CLI parses JSON and applies strict Zod validation; there is no authentication because the operator controls the local process (`src/cli.ts`, `src/provenance.ts`). +- Bootstrap CLI -> public manifest file/stdout: validated JSON crosses into a publishable local artifact. Credential-shaped allowlisted values are replaced, unknown keys fail, and redaction evidence is checked; the file is not signed by this path (`src/provenance.ts`). +- Repository/GitHub event -> hosted workflow: checked-out repository state and workflow inputs enter a GitHub-hosted runner. Workflow permissions are `contents: read` and `id-token: write`; action dependencies are immutable pins on current main (`.github/workflows/ai-attestation-reusable.yml`). +- Hosted workflow -> Sigstore: cosign exchanges GitHub OIDC identity for keyless signing material over the network. Verification constrains the OIDC issuer and repository workflow identity (`.github/workflows/ai-attestation-reusable.yml`). +- Hosted workflow -> GitHub artifact storage -> verification job: JSON and cosign bundle cross an external artifact-storage boundary. The verification job downloads and verifies them before success (`.github/workflows/ai-attestation-reusable.yml`). +- Bootstrap notification plan -> GitHub/webhook: governed summaries cross an authenticated API or HTTPS boundary. Credential filtering, GitHub target parsing, and webhook destination controls are implemented separately from provenance (`src/notifications.ts`, `src/github/client.ts`). +- Future private bundle -> encrypted logical sink: sensitive evidence would cross into remote storage using short-lived identity. This boundary is unimplemented and prohibited from production use in the current scope. + +#### Diagram + +```mermaid +flowchart LR + Operator["Operator or CI input"] --> CLI["Bootstrap provenance CLI"] + CLI --> Validator["Strict schema and redaction"] + Validator --> PublicFile["Public manifest"] + Repo["Repository and GitHub event"] --> Runner["Hosted attestation workflow"] + Runner --> Sigstore["Keyless signing service"] + Runner --> Artifacts["GitHub artifact storage"] + Artifacts --> Verify["Verification job"] + PrivateInput["Future private evidence"] --> DisabledSink["Disabled remote sink"] +``` + +## Assets and security objectives + +| Asset | Why it matters | Security objective (C/I/A) | +|---|---|---| +| Credentials and identity tokens | Exposure permits repository, cloud, or signing impersonation | C, I | +| Private model, prompt, tool, and customer evidence | May contain proprietary or personal information | C | +| Public provenance manifest | Consumers rely on subject, execution, and reviewer lineage | I, A | +| Cosign bundle and OIDC identity | Establish who signed which commit-scoped evidence | I | +| Reviewer lineage | Drives material-change governance and auditability | I | +| Workflow and immutable action pins | Compromise can forge or leak all downstream evidence | I, C, A | +| Artifact and future sink retention | Evidence must remain available without unauthorized mutation or reads | I, A, C | +| Audit records | Needed to detect improper capture, access, deletion, or verification | I, A | + +## Attacker model + +### Capabilities + +- A pull-request author can control repository content and some workflow-visible metadata, but should not receive protected repository secrets. +- A malicious or mistaken operator can supply arbitrary JSON to the local provenance CLI and choose input/output paths. +- A compromised dependency or workflow revision can execute in CI with that job's granted permissions. +- An attacker who compromises future sink write credentials may attempt overwrite, deletion, or ciphertext substitution; a read-role compromise may expose private evidence. + +### Non-capabilities + +- A normal pull-request author cannot mint the repository's GitHub OIDC identity outside an authorized workflow. +- The local CLI is not a network service and exposes no listener or multi-tenant request surface. +- No production private sink, encryption-key service, or audited read path exists in current code; threats to those are design requirements, not claims of present exposure. +- Physical storage administration and organization-owner compromise are outside this repository-level model. + +## Entry points and attack surfaces + +| Surface | How reached | Trust boundary | Notes | Evidence (repo path / symbol) | +|---|---|---|---|---| +| Provenance input JSON | `bootstrap provenance create --input` | Local file -> CLI | Arbitrary JSON; strict parsing must fail closed | `src/cli.ts`, `createPublicProvenance` | +| Public manifest validation | `bootstrap provenance validate --input` | Local file -> validator | Hand-authored manifests must not bypass redaction evidence | `src/cli.ts`, `validatePublicProvenance` | +| Public output path | `--output` | CLI -> local filesystem | Operator-controlled path; command is intentionally mutating | `src/cli.ts` provenance command | +| Workflow inputs | Reusable workflow caller | Repository/workflow caller -> hosted runner | Provider, model, prompt hash, artifact name, and retention enter shell/YAML contexts | `.github/workflows/ai-attestation-reusable.yml` | +| GitHub OIDC | `cosign sign-blob` | Hosted runner -> Sigstore | Short-lived identity; repository workflow identity is verified | `.github/workflows/ai-attestation-reusable.yml` | +| Artifact upload/download | GitHub Actions artifact service | Runner -> external storage -> verifier | Integrity depends on signature verification and correct subject binding | `.github/workflows/ai-attestation-reusable.yml` | +| Notification destinations | Material-action delivery | CLI -> GitHub API/webhook | Shares secret detection but is a separate public-output surface | `src/notifications.ts`, `src/github/client.ts` | +| Generated workflows | Bootstrap renderer/apply | Manifest -> managed repository files | Unsafe projection could replicate a provenance weakness fleet-wide | `src/archetypes.ts`, `src/render.ts` | + +## Top abuse paths + +1. Sensitive-data disclosure: attacker or operator places private text in a generic metadata field -> generator copies it -> public artifact is uploaded -> private material becomes durable and searchable. +2. Secret leakage outside metadata: credential is placed in workflow/ref/reviewer identity -> validation scans only metadata -> public manifest publishes the credential. +3. Reviewer spoofing: caller supplies an `approved` reviewer object -> local schema validates shape without querying GitHub -> downstream policy mistakes user input for authenticated approval. +4. Subject substitution: a valid manifest is generated for one commit -> filename or surrounding workflow labels it as another -> consumer verifies format but not a signature bound to the expected subject. +5. Workflow supply-chain compromise: mutable or compromised action executes with OIDC permission -> attacker signs false evidence or exfiltrates workflow data -> verification trusts the authorized workflow identity. +6. Artifact replacement or verification confusion: attacker substitutes JSON/bundle pairs or exploits broad certificate identity matching -> verification succeeds for unintended workflow evidence. +7. Capture fail-open: provenance generation, signing, encryption, or sink write fails -> merge gate treats absence as optional -> material change lands without required evidence. +8. Future private-sink compromise: write role overwrites or deletes ciphertext, or read role is overbroad -> evidence integrity/availability or confidentiality is lost without an audit signal. + +## Threat model table + +| Threat ID | Threat source | Prerequisites | Threat action | Impact | Impacted assets | Existing controls (evidence) | Gaps | Recommended mitigations | Detection ideas | Likelihood | Impact severity | Priority | +|---|---|---|---|---|---|---|---|---|---|---|---|---| +| TM-001 | Malicious input or operator error | Sensitive content reaches public generation | Smuggle secrets or private text through permitted fields | Public disclosure and credential compromise | Credentials, private evidence | Credential patterns and typed placeholders (`src/provenance.ts`) | Pattern matching cannot classify arbitrary private data | Strict metadata key allowlist; bounds and strict objects; prohibit logs/prompts/tool output; adversarial fixtures | Alert on redaction count; scan generated public artifacts before upload | Medium | High | High | +| TM-002 | Malicious input | Attacker controls non-metadata identity fields | Place credential-like text in ref, workflow, URL, or lineage | Secret appears in public output | Credentials, public manifest | Strong SHA/run-id formats (`src/provenance.ts`) | Previous scanning covered metadata only | Apply credential rejection and length bounds to every free-text public field | Negative fixtures for each field; artifact secret scan | Medium | High | High | +| TM-003 | PR author or compromised caller | Reviewer data is supplied by caller | Claim an approval that GitHub never authenticated | Material changes appear independently reviewed | Reviewer lineage, public manifest | Reviewer state is typed (`src/provenance.ts`) | No GitHub review retrieval or head-SHA binding | Resolve reviews through least-privilege GitHub API; bind reviewer, state, submitted SHA, and author separation | Compare manifest lineage with GitHub API during verification | High | High | High | +| TM-004 | Artifact or local-file attacker | Consumer accepts unsigned public CLI output | Modify subject, execution, or lineage after generation | Forged provenance accepted | Public manifest | Separate AI workflow signs a commit-scoped blob (`.github/workflows/ai-attestation-reusable.yml`) | CLI public schema and signed AI blob are not unified | Canonical serialization; sign public manifest; verify exact repository, workflow, ref, and commit expectations | Record verification failures and signer identity | Medium | High | High | +| TM-005 | Dependency or workflow attacker | CI dependency or workflow permissions are compromised | Mint authorized-looking false evidence or leak inputs | Fleet-wide integrity or confidentiality loss | Workflow, OIDC identity, evidence | Immutable action pins and narrow permissions (`.github/workflows/ai-attestation-reusable.yml`) | Certificate identity regex is broad; workflow source lineage is not embedded in public schema | Pin all dependencies; restrict certificate identity to exact workflow/ref; include workflow digest and reusable-workflow SHA | Monitor pin drift and unexpected signer identities | Low | High | High | +| TM-006 | Infrastructure failure or policy bypass | Capture/sign/encrypt/store step fails | Treat missing provenance as non-blocking | Material merge lacks required audit evidence | Evidence availability, governance | `provenance-capture-failure` is a material action (`src/notifications.ts`) | No required merge gate consumes end-to-end capture result | Add fail-closed required check for material changes with explicit non-material classification | Alert on absent/failed capture and bypass attempts | Medium | High | High | +| TM-007 | Future sink writer/read-role attacker | Remote sink is enabled with excessive privileges | Read private bundles or overwrite/delete evidence | Disclosure or loss of audit integrity | Private evidence, retention, audit records | Confirmed design requires OIDC and role separation | Adapter, encryption, Object Lock, and audited reads do not exist | Envelope encryption; separate read/write roles; immutable retention; deny listing where possible; audited break-glass reads | Cloud audit logs, write-once checks, unexpected read alerts | Low until enabled | High | Medium | +| TM-008 | Operator or local attacker | CLI runs with filesystem privileges | Choose an unintended output path and overwrite a file | Local integrity loss | Local repository files | Explicit required `--output`; normal OS permissions (`src/cli.ts`) | No confinement to `provenance/runs` | Add optional safe-root enforcement for CI use; retain explicit override only for interactive operator use | Log resolved output path; CI assertion on destination | Low | Medium | Low | + +## Criticality calibration + +- Critical: a broadly exploitable path to organization credentials or private evidence across many repositories; forged signed provenance that directly authorizes releases; compromise of production encryption keys plus immutable evidence. None is established in current local-only scope. +- High: public leakage of a credential/private payload; authenticated reviewer or subject forgery that permits a material merge; workflow/OIDC compromise that signs false evidence. +- Medium: future sink availability loss with recovery, narrow exposure requiring a privileged role, or capture outage that blocks work but does not fail open. +- Low: operator-only local overwrite with normal filesystem access, malformed input rejected before output, or noisy availability impact limited to a rerunnable local command. + +## Focus paths for security review + +| Path | Why it matters | Related Threat IDs | +|---|---|---| +| `src/provenance.ts` | Defines the public schema, allowlist, redaction, and validation evidence | TM-001, TM-002, TM-003, TM-004 | +| `tests/provenance.test.ts` | Holds adversarial fixtures that prevent fail-open regression | TM-001, TM-002, TM-004 | +| `src/cli.ts` | Parses untrusted local JSON and writes operator-selected paths | TM-001, TM-008 | +| `.github/workflows/ai-attestation-reusable.yml` | Holds OIDC, cosign, artifact, and verification boundaries | TM-004, TM-005, TM-006 | +| `.github/workflows/ai-attestation.yml` | Defines the repository caller and permissions | TM-005, TM-006 | +| `src/archetypes.ts` | Replicates workflow behavior into downstream repositories | TM-005, TM-006 | +| `src/manifest.ts` | Accepts workflow identity, provider/model, artifact, and retention configuration | TM-001, TM-005 | +| `src/notifications.ts` | Shares secret filtering and represents capture failure as a material action | TM-001, TM-006 | +| `src/github/client.ts` | Likely integration point for authenticated reviewer-lineage resolution | TM-003 | +| `scripts/ci/check-pr-governance.sh` | Current authenticated reviewer and material-change gate logic | TM-003, TM-006 | +| `scripts/ci/check-action-pins.sh` | Enforces workflow supply-chain immutability | TM-005 | +| `src/render.ts` | Applies generated provenance workflow changes across managed repositories | TM-005, TM-006 | + +Quality check: local CLI and CI entry points are separated; every identified trust boundary appears in at least one threat; current and future controls are distinguished; the confirmed privacy, OIDC, role-separation, review, and production-sink assumptions are reflected; unresolved provider, retention, reviewer-source, and merge-gate decisions remain explicit. diff --git a/src/provenance.ts b/src/provenance.ts index af8ba2a..abdffd2 100644 --- a/src/provenance.ts +++ b/src/provenance.ts @@ -1,6 +1,6 @@ import { z } from "zod"; -export const PUBLIC_PROVENANCE_SCHEMA_VERSION = 1; +export const PUBLIC_PROVENANCE_SCHEMA_VERSION = 2; export const REDACTED_CREDENTIAL = "[REDACTED:CREDENTIAL]"; const shaPattern = /^[0-9a-f]{40}$/; @@ -10,49 +10,178 @@ const credentialPatterns = [ /-----BEGIN (?:[A-Z ]+ )?PRIVATE KEY-----/, /\b(?:api[_-]?key|access[_-]?token|password|secret|token)\s*[:=]\s*[^\s]+/i ]; +export const LEGACY_PUBLIC_PROVENANCE_SCHEMA_VERSION = 1; +export const CURRENT_PUBLIC_PROVENANCE_SCHEMA_VERSION = PUBLIC_PROVENANCE_SCHEMA_VERSION; +const MAX_PUBLIC_METADATA_INPUT_LENGTH = 512; +const MAX_PUBLIC_METADATA_OUTPUT_LENGTH = MAX_PUBLIC_METADATA_INPUT_LENGTH * 4; -const reviewerSchema = z.object({ +const legacyReviewerSchema = z.object({ login: z.string().min(1), state: z.enum(["approved", "commented", "changes_requested"]) -}); +}).strict(); -export const publicProvenanceSchema = z +export const legacyPublicProvenanceSchema = z .object({ - schemaVersion: z.literal(PUBLIC_PROVENANCE_SCHEMA_VERSION), + schemaVersion: z.literal(LEGACY_PUBLIC_PROVENANCE_SCHEMA_VERSION), runId: z.string().regex(/^[A-Za-z0-9._-]+$/), subject: z.object({ repository: z.string().regex(/^[^/\s]+\/[^/\s]+$/), commitSha: z.string().regex(shaPattern), ref: z.string().min(1) - }), + }).strict(), execution: z.object({ workflow: z.string().min(1), runUrl: z.string().url().optional(), createdAt: z.string().datetime({ offset: true }) - }), - reviewers: z.array(reviewerSchema), + }).strict(), + reviewers: z.array(legacyReviewerSchema), metadata: z.record(z.string(), z.string()), redaction: z.object({ policyVersion: z.literal(1), replacements: z.number().int().nonnegative() - }) + }).strict() }) + .strict() .superRefine((manifest, context) => { - for (const [key, value] of Object.entries(manifest.metadata)) { + const publicStrings: Array<[string, string]> = [ + ["runId", manifest.runId], + ["subject.repository", manifest.subject.repository], + ["subject.ref", manifest.subject.ref], + ["execution.workflow", manifest.execution.workflow] + ]; + if (manifest.execution.runUrl) publicStrings.push(["execution.runUrl", manifest.execution.runUrl]); + manifest.reviewers.forEach((reviewer, index) => publicStrings.push([`reviewers.${index}.login`, reviewer.login])); + Object.entries(manifest.metadata).forEach(([key, value]) => { + publicStrings.push([`metadata key ${key}`, key], [`metadata.${key}`, value]); + }); + for (const [path, value] of publicStrings) { if (containsCredential(value)) { context.addIssue({ code: "custom", - message: `Public provenance metadata ${key} contains a credential-like literal. Redact it before validation.` + message: `Legacy public provenance field ${path} contains a credential-like literal.` }); } } }); -export type PublicProvenance = z.infer; -export type PublicProvenanceInput = Omit & { - metadata?: Record; +const publicText = (label: string, maximumLength: number) => + z + .string() + .min(1) + .max(maximumLength) + .refine((value) => !containsCredential(value), `${label} contains a credential-like literal.`); + +const reviewerSchema = z.object({ + login: z + .string() + .max(100) + .regex(/^[A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?(?:\[bot\])?$/) + .refine((value) => !containsCredential(value), "Reviewer login contains a credential-like literal."), + state: z.enum(["approved", "commented", "changes_requested"]) +}).strict(); + +const subjectSchema = z.object({ + repository: z + .string() + .regex(/^[^/\s]+\/[^/\s]+$/) + .max(201) + .refine((value) => !containsCredential(value), "Subject repository contains a credential-like literal."), + commitSha: z.string().regex(shaPattern), + ref: publicText("Subject ref", 256) +}).strict(); + +const executionSchema = z.object({ + workflow: publicText("Workflow name", 200), + runUrl: z.string().url().max(2_048).refine((value) => !containsCredential(value), "Run URL contains a credential-like literal.").optional(), + createdAt: z.string().datetime({ offset: true }) +}).strict(); + +const metadataInputValue = z + .string() + .min(1) + .max(MAX_PUBLIC_METADATA_INPUT_LENGTH) + .refine( + (value) => !value.includes(REDACTED_CREDENTIAL), + "Public provenance input must not contain the reserved redaction placeholder." + ); +const metadataOutputValue = publicText("Public provenance metadata", MAX_PUBLIC_METADATA_OUTPUT_LENGTH); +const metadataInputShape = { + policy: metadataInputValue.optional(), + generator: metadataInputValue.optional(), + aiProvider: metadataInputValue.optional(), + aiModel: metadataInputValue.optional(), + promptHash: metadataInputValue.optional(), + changeClass: metadataInputValue.optional() +}; +const metadataOutputShape = { + policy: metadataOutputValue.optional(), + generator: metadataOutputValue.optional(), + aiProvider: metadataOutputValue.optional(), + aiModel: metadataOutputValue.optional(), + promptHash: metadataOutputValue.optional(), + changeClass: metadataOutputValue.optional() }; +export const PUBLIC_PROVENANCE_METADATA_KEYS = [ + "policy", + "generator", + "aiProvider", + "aiModel", + "promptHash", + "changeClass" +] as const; + +export const publicProvenanceMetadataSchema = z.object(metadataOutputShape).strict(); + +export const publicProvenanceInputSchema = z.object({ + runId: z + .string() + .max(128) + .regex(/^[A-Za-z0-9._-]+$/) + .refine((value) => !containsCredential(value), "Run ID contains a credential-like literal."), + subject: subjectSchema, + execution: executionSchema, + reviewers: z.array(reviewerSchema), + metadata: z.object(metadataInputShape).strict().optional() +}).strict(); + +export const publicProvenanceSchema = z + .object({ + schemaVersion: z.literal(CURRENT_PUBLIC_PROVENANCE_SCHEMA_VERSION), + runId: z + .string() + .max(128) + .regex(/^[A-Za-z0-9._-]+$/) + .refine((value) => !containsCredential(value), "Run ID contains a credential-like literal."), + subject: subjectSchema, + execution: executionSchema, + reviewers: z.array(reviewerSchema), + metadata: publicProvenanceMetadataSchema, + redaction: z.object({ + policyVersion: z.literal(1), + replacements: z.number().int().nonnegative() + }).strict() + }) + .strict() + .superRefine((manifest, context) => { + const recordedReplacements = Object.values(manifest.metadata).reduce( + (total, value) => total + countOccurrences(value ?? "", REDACTED_CREDENTIAL), + 0 + ); + if (recordedReplacements !== manifest.redaction.replacements) { + context.addIssue({ + code: "custom", + path: ["redaction", "replacements"], + message: "Public provenance redaction evidence does not match the typed placeholders in metadata." + }); + } + }); + +export type PublicProvenance = z.infer; +export type LegacyPublicProvenance = z.infer; +export type SupportedPublicProvenance = PublicProvenance | LegacyPublicProvenance; +export type PublicProvenanceInput = z.input; + export function containsCredential(value: string): boolean { return credentialPatterns.some((pattern) => pattern.test(value)); } @@ -61,7 +190,8 @@ export function redactPublicText(value: string): { value: string; replacements: let redacted = value; let replacements = 0; for (const pattern of credentialPatterns) { - redacted = redacted.replace(pattern, () => { + const globalPattern = new RegExp(pattern.source, pattern.flags.includes("g") ? pattern.flags : `${pattern.flags}g`); + redacted = redacted.replace(globalPattern, () => { replacements += 1; return REDACTED_CREDENTIAL; }); @@ -70,22 +200,24 @@ export function redactPublicText(value: string): { value: string; replacements: } export function createPublicProvenance(input: PublicProvenanceInput): PublicProvenance { + const parsedInput = publicProvenanceInputSchema.parse(input); let replacements = 0; const metadata = Object.fromEntries( - Object.entries(input.metadata ?? {}).map(([key, rawValue]) => { - if (rawValue === undefined) return [key, ""]; - const redacted = redactPublicText(rawValue); - replacements += redacted.replacements; - return [key, redacted.value]; - }) + Object.entries(parsedInput.metadata ?? {}) + .filter((entry): entry is [string, string] => entry[1] !== undefined) + .map(([key, rawValue]) => { + const redacted = redactPublicText(rawValue); + replacements += redacted.replacements; + return [key, redacted.value]; + }) ); return publicProvenanceSchema.parse({ - schemaVersion: PUBLIC_PROVENANCE_SCHEMA_VERSION, - runId: input.runId, - subject: input.subject, - execution: input.execution, - reviewers: input.reviewers, + schemaVersion: CURRENT_PUBLIC_PROVENANCE_SCHEMA_VERSION, + runId: parsedInput.runId, + subject: parsedInput.subject, + execution: parsedInput.execution, + reviewers: parsedInput.reviewers, metadata, redaction: { policyVersion: 1, replacements } }); @@ -94,3 +226,11 @@ export function createPublicProvenance(input: PublicProvenanceInput): PublicProv export function validatePublicProvenance(value: unknown): PublicProvenance { return publicProvenanceSchema.parse(value); } + +export function readLegacyPublicProvenance(value: unknown): LegacyPublicProvenance { + return legacyPublicProvenanceSchema.parse(value); +} + +function countOccurrences(value: string, needle: string): number { + return value.split(needle).length - 1; +} diff --git a/tests/provenance.test.ts b/tests/provenance.test.ts index f7c59b4..62cc1c1 100644 --- a/tests/provenance.test.ts +++ b/tests/provenance.test.ts @@ -1,6 +1,15 @@ import { describe, expect, it } from "vitest"; -import { REDACTED_CREDENTIAL, createPublicProvenance, validatePublicProvenance } from "../src/provenance.js"; +import { + CURRENT_PUBLIC_PROVENANCE_SCHEMA_VERSION, + LEGACY_PUBLIC_PROVENANCE_SCHEMA_VERSION, + PUBLIC_PROVENANCE_METADATA_KEYS, + REDACTED_CREDENTIAL, + createPublicProvenance, + publicProvenanceMetadataSchema, + readLegacyPublicProvenance, + validatePublicProvenance +} from "../src/provenance.js"; const input = { runId: "12345.1", @@ -12,17 +21,22 @@ const githubPat = ["github", "pat"].join("_") + "_abcdefghijklmnopqrstuvwxyz1234 const awsAccessKey = ["AK", "IA"].join("") + "ABCDEFGHIJKLMNOP"; describe("public provenance", () => { + it("keeps the exported metadata allowlist synchronized with the schema", () => { + expect(Object.keys(publicProvenanceMetadataSchema.shape)).toEqual(PUBLIC_PROVENANCE_METADATA_KEYS); + }); + it("redacts adversarial credential literals before creating a public manifest", () => { const provenance = createPublicProvenance({ ...input, metadata: { - trace: githubPat, - cloud: awsAccessKey, - configured: "token=should-not-escape" + policy: githubPat, + generator: awsAccessKey, + aiProvider: ["token", "should-not-escape"].join("=") } }); - expect(provenance.metadata).toEqual({ trace: REDACTED_CREDENTIAL, cloud: REDACTED_CREDENTIAL, configured: REDACTED_CREDENTIAL }); + expect(provenance.metadata).toEqual({ policy: REDACTED_CREDENTIAL, generator: REDACTED_CREDENTIAL, aiProvider: REDACTED_CREDENTIAL }); + expect(provenance.schemaVersion).toBe(CURRENT_PUBLIC_PROVENANCE_SCHEMA_VERSION); expect(provenance.redaction.replacements).toBe(3); expect(JSON.stringify(provenance)).not.toContain("should-not-escape"); }); @@ -30,9 +44,9 @@ describe("public provenance", () => { it("rejects a hand-authored public manifest containing a credential-like literal", () => { expect(() => validatePublicProvenance({ ...input, - schemaVersion: 1, - metadata: { leaked: "password=not-for-publication" }, - redaction: { policyVersion: 1, replacements: 0 } + schemaVersion: CURRENT_PUBLIC_PROVENANCE_SCHEMA_VERSION, + metadata: { policy: ["password", "not-for-publication"].join("=") }, + redaction: { policyVersion: 1, replacements: 1 } })).toThrow("credential-like literal"); }); @@ -42,4 +56,125 @@ describe("public provenance", () => { expect(provenance.reviewers).toEqual(input.reviewers); expect(provenance.metadata.policy).toBe("public-repository-standard-v1"); }); + + it("rejects metadata outside the public allowlist without echoing its value", () => { + const unknownValue = "internal customer material"; + let message = ""; + + try { + createPublicProvenance({ ...input, metadata: { privateTrace: unknownValue } } as never); + } catch (error) { + message = error instanceof Error ? error.message : String(error); + } + + expect(message).toContain("Unrecognized key"); + expect(message).toContain("privateTrace"); + expect(message).not.toContain(unknownValue); + }); + + it("rejects unknown fields at every public schema boundary", () => { + expect(() => validatePublicProvenance({ + ...input, + schemaVersion: CURRENT_PUBLIC_PROVENANCE_SCHEMA_VERSION, + subject: { ...input.subject, unexpected: "not public" }, + metadata: {}, + redaction: { policyVersion: 1, replacements: 0 } + })).toThrow("Unrecognized key"); + }); + + it("redacts every repeated credential literal and records exact evidence", () => { + const provenance = createPublicProvenance({ + ...input, + metadata: { generator: `${githubPat} ${githubPat}` } + }); + + expect(provenance.metadata.generator).toBe(`${REDACTED_CREDENTIAL} ${REDACTED_CREDENTIAL}`); + expect(provenance.redaction.replacements).toBe(2); + }); + + it("rejects forged redaction counts", () => { + expect(() => validatePublicProvenance({ + ...input, + schemaVersion: CURRENT_PUBLIC_PROVENANCE_SCHEMA_VERSION, + metadata: { generator: REDACTED_CREDENTIAL }, + redaction: { policyVersion: 1, replacements: 0 } + })).toThrow("redaction evidence does not match"); + }); + + it("rejects credential-like literals outside redactable metadata", () => { + expect(() => createPublicProvenance({ + ...input, + execution: { ...input.execution, workflow: ["token", "not-public"].join("=") } + })).toThrow("Workflow name contains a credential-like literal"); + }); + + it.each([ + ["run ID", { ...input, runId: awsAccessKey }], + ["repository", { ...input, subject: { ...input.subject, repository: `acme/${awsAccessKey}` } }], + ["reviewer", { ...input, reviewers: [{ login: awsAccessKey, state: "approved" as const }] }] + ])("rejects credential-like literals in the %s identity", (_label, unsafeInput) => { + expect(() => createPublicProvenance(unsafeInput)).toThrow("credential-like literal"); + }); + + it("preserves valid GitHub App reviewer logins", () => { + const provenance = createPublicProvenance({ + ...input, + reviewers: [{ login: "dependabot[bot]", state: "approved" }] + }); + + expect(provenance.reviewers).toEqual([{ login: "dependabot[bot]", state: "approved" }]); + }); + + it("rejects a pre-supplied reserved redaction placeholder", () => { + expect(() => createPublicProvenance({ + ...input, + metadata: { generator: REDACTED_CREDENTIAL } + })).toThrow("reserved redaction placeholder"); + }); + + it("continues to validate legacy version-1 manifests explicitly", () => { + const legacyInput = { + ...input, + schemaVersion: LEGACY_PUBLIC_PROVENANCE_SCHEMA_VERSION, + metadata: { trace: "public-build-trace" }, + redaction: { policyVersion: 1, replacements: 0 } + }; + const legacy = readLegacyPublicProvenance(legacyInput); + + expect(legacy.schemaVersion).toBe(LEGACY_PUBLIC_PROVENANCE_SCHEMA_VERSION); + expect(legacy.metadata).toEqual({ trace: "public-build-trace" }); + expect(() => validatePublicProvenance(legacyInput)).toThrow(); + }); + + it("rejects unknown fields in explicit legacy reads", () => { + expect(() => readLegacyPublicProvenance({ + ...input, + schemaVersion: LEGACY_PUBLIC_PROVENANCE_SCHEMA_VERSION, + subject: { ...input.subject, privatePayload: "not part of version 1" }, + metadata: {}, + redaction: { policyVersion: 1, replacements: 0 } + })).toThrow("Unrecognized key"); + }); + + it("rejects credential-like legacy metadata keys", () => { + expect(() => readLegacyPublicProvenance({ + ...input, + schemaVersion: LEGACY_PUBLIC_PROVENANCE_SCHEMA_VERSION, + metadata: { [githubPat]: "public-build-trace" }, + redaction: { policyVersion: 1, replacements: 0 } + })).toThrow("credential-like literal"); + }); + + it("supports worst-case credential redaction expansion within the output bound", () => { + const repeated = Array.from({ length: 60 }, () => ["token", "x"].join("=")).join(" "); + const provenance = createPublicProvenance({ + ...input, + metadata: { generator: repeated } + }); + + expect(provenance.redaction.replacements).toBe(60); + expect(provenance.metadata.generator).toBe( + Array.from({ length: 60 }, () => REDACTED_CREDENTIAL).join(" ") + ); + }); });