diff --git a/devlog/_plan/260807_compatibility_lab/010_cl10_public_evidence_export.md b/devlog/_plan/260807_compatibility_lab/010_cl10_public_evidence_export.md new file mode 100644 index 0000000000..2aadba8966 --- /dev/null +++ b/devlog/_plan/260807_compatibility_lab/010_cl10_public_evidence_export.md @@ -0,0 +1,771 @@ +# CL-10 - Public Evidence Export, Publishing, and Community Trust + +## Programme position + +**Repository:** `lidge-jun/opencodex` +**Integration target:** `dev` +**Branch:** `feat/cl-10-public-evidence-contract` +**Starting SHA:** `4fed8d3fe431ad23be83f3aff2af18ef8b8ecd71` +**CL-09 merge prerequisite:** satisfied by #1489 at `4fed8d3fe431ad23be83f3aff2af18ef8b8ecd71` + +CL-09 is merged. CL-10 is the final planned Compatibility Lab phase. + +This PR began contract-only and the contract was independently reviewed and accepted on 2026-08-12. CL-10.1 through CL-10.4 runtime implementation is now authorized on this branch. CL-10.5 remote publishing remains blocked until the exact transport/service contract in section 18 is independently accepted. + +--- + +# 1. Goal + +CL-10 answers: + +> How can a user deliberately export and, later, publish a narrowly allowlisted subset of Compatibility Lab evidence for community use without leaking installation-local identifiers, custom configuration, user data, credentials, or private operational metadata, and without letting untrusted community data silently affect local canonical verdicts or routing? + +The architecture is deliberately one-way at the local trust boundary: + +```text +Local canonical Lab evidence + | + | explicit export projection only + v +Public allowlist projector + | + +--> export privacy scan / fail closed + | + +--> export-scoped IDs + | + +--> optional public-export artifacts only + v +Canonical public bundle + | + +--> local preview/export + +--> explicit publish action, only after transport contract is accepted + v +Community bundle + | + +--> schema/digest/signature verification + +--> separate community trust/cache domain + | + X--> no write into local compatibility.jsonl + X--> no canonical verdict promotion/degradation + X--> no Routing Profile or Router Intelligence input + X--> no CL-08 scheduling input +``` + +CL-10 shares evidence. It does not transfer local authority. + +--- + +# 2. Existing authority carried forward + +CL-10 must preserve the existing CL-00 security/privacy contract, especially its `Local evidence versus public export` boundary: + +- public export uses a new allowlist-only schema; +- local subject/event/artifact IDs are replaced with export-scoped opaque IDs; +- endpoint and provider-instance fingerprints are omitted; +- local request, decision, and Fabric references are omitted; +- precise local paths, custom headers, project/location, account context, local errors, and raw latency traces are omitted; +- custom provider/model names are private by default; +- artifact bytes are exportable only when their policy explicitly allows `public_export`; +- export-specific secret/PII scanning is mandatory; +- unknown fields fail closed. + +CL-10 may tighten those rules. It must not weaken them silently. + +--- + +# 3. Hard CL-10 invariants + +CL-10 V1 must guarantee: + +```text +0 automatic telemetry upload +0 background publishing without an explicit user action +0 export of local subject/event/artifact/request/decision/Fabric identifiers +0 export of endpoint/provider-instance/custom-header/project/location fingerprints +0 export of credentials, account identity, prompts, responses, tool payloads, repository data, paths, or hidden reasoning +0 export of custom provider/model names unless a later reviewed public-registry authority explicitly permits them +0 community bundle writes into compatibility.jsonl +0 community evidence promotion/degradation of canonical local verdicts +0 community evidence influence on Routing Profiles or Router Intelligence +0 community evidence influence on CL-08 scheduling +0 combined local/community compatibility score +``` + +Export, publish, import, verification, or community-cache failure must not affect normal production request execution. + +--- + +# 4. Chosen approach + +Three approaches were considered. + +## 4.1 Chosen: deterministic public projection plus separate community trust domain + +Project local evidence into a new public schema containing only export-safe fields. Produce a canonical bundle with a digest and publisher signature. Community imports are verified and stored outside the local canonical evidence authority. + +Benefits: + +- privacy boundary is explicit and machine-testable; +- exported bytes are reproducible from the same local evidence and export policy; +- local IDs never leave the installation; +- community provenance can be verified without treating publisher claims as canonical truth; +- imported evidence cannot contaminate local verdicts or routing. + +## 4.2 Rejected: publish local Lab JSONL or SQLite rows directly + +The local schemas contain installation-scoped identifiers and fields whose local visibility does not imply public-export permission. Direct publication would make privacy depend on callers remembering ad-hoc redaction rules. + +## 4.3 Rejected: remote service as canonical evidence authority + +A hosted service may aggregate public bundles later, but it must not become the canonical authority for local Lab verdicts. OpenCodex must remain able to reproduce local verdicts from local canonical evidence without network access. + +--- + +# 5. Public exportability gate + +An observation is exportable only when all required public identity fields can be represented without private configuration. + +V1 exportable routes are limited to entries in the repo-reviewed `PublicRouteRegistryManifestV1` whose exported behavior identity is entirely composed from reviewed public fields. + +The public-route authority is a versioned, content-addressed repository artifact owned by OpenCodex, not a publisher-supplied assertion: + +```ts +interface PublicRouteRegistryManifestV1 { + schemaVersion: "public_route_registry_v1"; + registryVersion: string; + sourceCommit: string; + entries: PublicRouteRegistryEntryV1[]; + manifestDigest: string; +} + +interface PublicRouteRegistryEntryV1 { + providerId: string; + modelId: string; + adapterFamilies: Array<"openai-responses" | "openai-chat" | "anthropic-messages">; +} +``` + +CL-10.1 must ship and validate this manifest before any route-scoped record is exportable. The manifest may be updated only by reviewed repository changes with a new digest/version. Dynamic model discovery, cached catalogs, user configuration, imported bundles, and a matching spelling alone can never extend this authority. + +A route is not exportable when any behavior-relevant identity depends on a private/custom value, including: + +- custom provider instance or custom provider name; +- custom model ID or alias not in the reviewed public registry authority; +- non-default/custom endpoint identity; +- private/custom header behavior; +- project, location, tenant, deployment, organization, or account context; +- private-network destination behavior; +- any other local behavior fingerprint that cannot be represented publicly without weakening exact-route semantics. + +Failing this gate is `not_exportable`, not an error and not a compatibility verdict. + +CL-10 must never broaden exact local evidence into a more general public claim merely by dropping private route dimensions. + +--- + +# 6. Public evidence schema + +CL-10 introduces `PublicEvidenceBundleV1` as a closed, versioned, allowlist-only schema. + +Conceptually: + +```ts +interface PublicEvidenceBundleV1 { + schemaVersion: "public_evidence_bundle_v1"; + exportPolicyVersion: "public_export_policy_v1"; + bundleId: string; + createdDayUtc: string; + publisher: PublicPublisherV1; + records: PublicEvidenceRecordV1[]; + artifacts: PublicArtifactV1[]; + bundleDigest: string; + signature: PublicBundleSignatureV1; +} + +interface PublicEvidenceRecordV1 { + recordId: string; + subjectId: string; + evidenceLayer: "protocol_conformance" | "live_route_compatibility" | "task_effectiveness"; + suiteId: string; + suiteVersion: string; + scenarioId: string; + scenarioVersion: string; + verdict: "CLAIMED" | "PROBED" | "VERIFIED" | "DEGRADED" | "UNSUPPORTED" | "BLOCKED" | "UNKNOWN"; + observedDayUtc: string; + subject: PublicEvidenceSubjectV1; + assertions: PublicAssertionSummaryV1[]; + incidentRefs?: PublicIncidentRefV1[]; + artifactRefs?: string[]; +} + +type PublicEvidenceSubjectV1 = + | PublicProtocolSubjectV1 + | PublicRouteSubjectV1 + | PublicTaskSubjectV1; + +interface PublicIncidentRefV1 { + corpusId: string; // exact reviewed `IC-NNN` identifier only +} +``` + +The public runtime types are dedicated CL-10 types. They may import closed scalar unions such as the existing verdict/evidence-layer literals, but they must not alias, extend, spread, or serialize local ledger/query DTO interfaces. A compile-time TypeScript shape is not the security boundary: every export/import path must pass the dedicated runtime validator for the matching public schema version. + +`PublicEvidenceSubjectV1` is layer-matched: protocol records use only a public protocol descriptor, live-route records use only a public route descriptor backed by `PublicRouteRegistryManifestV1`, and task records use a public task descriptor that nests the same public route descriptor plus reviewed public task/verifier authority fields. A layer/subject-kind mismatch is `schema_rejected`. + +Unknown top-level or nested fields fail export and import validation. + +`incidentRefs` contain only exact reviewed corpus identifiers matching `^IC-[0-9]{3}$` that exist in the repository incident authority. They never contain the corpus entry's historical issue URLs, devlog paths, test paths, prose, or source metadata. `artifactRefs` contain only public artifact IDs present in the same bundle; local artifact digests/relative paths are forbidden. + +--- + +# 7. Export-scoped identity + +Local identifiers must never be serialized into a public bundle. + +`bundleId`, `recordId`, `subjectId`, and public artifact IDs are derived only from canonical export-safe bytes under explicit domain-separated SHA-256 inputs. They must have no reversible or keyed relationship to: + +- local `RouteSubjectV1.subjectId`; +- local observation/event IDs; +- local artifact digests when the artifact is not explicitly public-exportable; +- request IDs; +- route decision IDs; +- Fabric/task references; +- installation salt. + +A public subject ID may be deterministic across publishers only from fields that are already public in `PublicRouteDescriptorV1`. It must never include or hash a private local dimension. + +--- + +# 8. Public route descriptor + +`PublicRouteDescriptorV1` contains only reviewed public registry identity and protocol behavior needed to interpret a community record. + +At minimum it may contain: + +```ts +interface PublicRouteDescriptorV1 { + providerId: string; + modelId: string; + adapterFamily: "openai-responses" | "openai-chat" | "anthropic-messages"; + compatibilityVersion: string; +} +``` + +`providerId` and `modelId` must come from an explicit public-registry allowlist. A configured value matching the spelling of a public ID is insufficient if the effective route uses private behavior dimensions that make the public claim ambiguous. + +No endpoint, headers, project/location, provider-instance identifier, account identifier, credential class, quota plan, or private capability fingerprint is included. + +--- + +# 9. Time and diagnostic minimization + +Public records use UTC day buckets (`YYYY-MM-DD`), not precise local timestamps. + +V1 exports no raw request latency, token timing, transport phase trace, provider error message, local error code, or local failure string. + +Assertion summaries must use scenario-defined closed assertion IDs and bounded result enums. They must not contain arbitrary observed strings. + +If an existing scenario assertion cannot be represented without free-form/private output, that assertion is omitted only when the scenario contract permits a complete public summary without it; otherwise the record is `not_exportable`. + +--- + +# 10. Public artifact policy + +Local artifact visibility does not imply public-export permission. + +An artifact may appear in a public bundle only when all are true: + +1. its producer/scenario policy explicitly marks the artifact class `public_export`; +2. bytes are already synthetic/sanitized under Lab artifact rules; +3. CL-10 performs a second export-specific sanitizer and secret/PII scan; +4. the artifact satisfies public bundle size/type limits; +5. the public artifact digest is computed from the final exported bytes, not copied from a private/local reference by assumption. + +V1 does not export arbitrary text logs, provider errors, traces containing timing detail, task patches, terminal logs, repository content, or raw request/response shapes. + +--- + +# 11. Export privacy scanner + +Before a bundle can be written as publishable, CL-10 must run an export-specific fail-closed validator. + +It must reject: + +- unknown fields; +- strings outside field-specific bounds; +- token/credential canaries; +- email/account/project/tenant identifiers; +- URLs, local paths, IP addresses where identifying, query strings, header-like material, or authorization values; +- local Lab IDs and known request/decision/Fabric ID formats; +- custom provider/model identifiers; +- precise timestamps where only day buckets are allowed; +- artifact bytes not explicitly marked `public_export`. + +`bun run privacy:scan` remains defense in depth and is not a substitute for this validator. + +--- + +# 12. Consent and user control + +There is no automatic export or publishing. + +V1 user flow must be explicit: + +```text +select export scope + -> generate local preview + -> show included record/artifact counts and excluded/not_exportable counts + -> explicit export action + -> local canonical bundle + -> optional explicit publish action only if a publish transport is authorized +``` + +Generating a preview performs no network request. + +A publish action must require an explicit user action for the specific bundle. CL-10 V1 must not introduce an always-on telemetry toggle, silent background upload, startup upload, or production-request-path upload. + +Deleting local Lab data remains absolute locally. Public copies already distributed cannot be cryptographically erased, so revocation semantics are required separately. + +## Sensitive purge interaction + +CL-00 sensitive purge remains authoritative over CL-10 local copies. A purge whose closed action set includes `export` must fail closed until every affected local export/staging copy is removed. CL-10 must additionally remove any locally-originated copy of an affected bundle that has been imported into the local `community/` cache. Third-party community bundles are unrelated to the local sensitive bytes and are not deleted merely because they contain the same public route identity. + +A local sensitive purge never waits for network access. If an affected bundle was previously published, CL-10 records or emits a bounded signed `privacy_retraction` revocation for its public bundle/record IDs when the reviewed transport is available, but remote acknowledgement is not a prerequisite for completing the mandatory local purge. The purge must not retain sensitive bytes merely to construct a later revocation. + +--- + +# 13. Publisher provenance and signatures + +A published bundle must be self-verifying for integrity and publisher continuity without exposing account identity. + +CL-10 V1 uses an installation-local Ed25519 publisher key created only when the user first requests a publishable bundle or publication. + +The private key: + +- lives outside JSONL, SQLite, artifacts, export bundles, and community cache; +- uses secret-file permissions; +- is never logged or exposed through API/UI/CLI output; +- is never used for route-subject identity or local verdict derivation. + +The public bundle contains: + +```ts +interface PublicPublisherV1 { + algorithm: "ed25519"; + keyId: string; + publicKey: string; +} + +interface PublicBundleSignatureV1 { + algorithm: "ed25519"; + signedDigest: string; + signature: string; +} +``` + +`keyId` is a domain-separated SHA-256 digest of the public key. + +A valid signature proves only that the same publisher key signed those exact canonical bytes. It does not prove the evidence is honest, representative, current, or trustworthy. + +## 13.1 Frozen canonical byte and signature contract + +CL-10 V1 uses RFC 8785 JSON Canonicalization Scheme (JCS) as the only canonical JSON representation. Canonical JSON bytes are UTF-8 bytes of the JCS string. Raw serialized imports must be valid UTF-8 JSON and must reject duplicate decoded object member names before semantic object construction. Duplicate detection is semantic after JSON string escape decoding, so `"a"` and `"\u0061"` are the same member name and must fail closed if both appear in one object. + +All public hash identities use the exact construction: + +```text +H(domain, value) = SHA-256(UTF8(domain) || 0x00 || UTF8(JCS(value))) +``` + +No trailing NUL is added. The exact V1 domain strings are: + +```text +subject ocx-lab-public:subject:v1 +record ocx-lab-public:record:v1 +bundle ocx-lab-public:bundle:v1 +bundle_digest ocx-lab-public:bundle-digest:v1 +artifact ocx-lab-public:artifact:v1 +publisher_key ocx-lab-public:publisher-key:v1 +revocation ocx-lab-public:revocation:v1 +route_registry ocx-lab-public:route-registry:v1 +``` + +The bundle identity preimages are frozen as semantic objects before JCS: + +```text +C = { + schemaVersion, + exportPolicyVersion, + createdDayUtc, + publisher, + records, + artifacts +} + +bundleId = H("ocx-lab-public:bundle:v1", C) + +bundleDigest = H( + "ocx-lab-public:bundle-digest:v1", + { ...C, bundleId } +) +``` + +Therefore `bundleId` is excluded from its own preimage, and both `bundleDigest` and `signature` are excluded from the `bundleId` preimage. `bundleDigest` includes the computed `bundleId`, but excludes both `bundleDigest` and `signature`. A bundle signature is exactly: + +```text +signature.algorithm = "ed25519" +signature.signedDigest = bundleDigest +signature.signature = Base64(Ed25519.Sign(privateKey, HexDecode(bundleDigest))) +``` + +The signature input is exactly the raw 32 bytes produced by hex-decoding the 64-character lowercase SHA-256 `bundleDigest`. There is no additional signature prefix because the signed digest is already domain-separated by `ocx-lab-public:bundle-digest:v1`. + +Publisher identity is exactly: + +```text +keyId = H( + "ocx-lab-public:publisher-key:v1", + { algorithm: "ed25519", publicKey } +) +``` + +where `publicKey` is the canonical Base64 representation of the Ed25519 SPKI DER bytes. + +Revocations use the same construction with a separate domain. After canonical sorting and duplicate rejection of targets: + +```text +R = { + schemaVersion, + issuedDayUtc, + publisher, + targets, + reason +} + +revocationId = H("ocx-lab-public:revocation:v1", R) +signature.signedDigest = revocationId +signature.signature = Base64(Ed25519.Sign(privateKey, HexDecode(revocationId))) +``` + +`revocationId` and `signature` are excluded from `R`. This binds schema/version, exact publisher identity, target bundle/record IDs, issued day, and finite reason under the dedicated revocation domain. + +Verification order is normative and exact for raw imported bundles: + +1. enforce the serialized byte ceiling; +2. require valid UTF-8 and reject duplicate decoded JSON object member names before object construction; +3. parse JSON and enforce nesting, array, object-key, and string bounds; +4. enforce the closed schema/version/field rules and recompute `publisher.keyId`; +5. recompute public subject/record/artifact identities, references, `bundleId`, and `bundleDigest` from canonical public-safe fields; +6. require `signature.signedDigest === bundleDigest`; +7. decode the canonical Ed25519 SPKI key and Base64 signature and verify Ed25519 over `HexDecode(bundleDigest)`; +8. validate repository-owned public-route, suite, scenario, verifier, and Fabric authority references; +9. for revocations, bootstrap authority only from an already-verified target bundle and require the exact publisher algorithm, `keyId`, and public key plus valid target membership before applying the revocation; +10. persist only after every preceding applicable check succeeds. + +A fixed test vector must lock these byte-level semantics so serializer, hash-domain, field-set, digest, or signing changes cannot silently create a second V1 wire format. + +--- + +# 14. Community trust model + +Imported community evidence is a separate trust class: `community_untrusted_v1`. + +Verification checks: + +- closed schema version; +- size/structure limits; +- canonical bundle digest; +- publisher signature; +- public route allowlist; +- scenario/suite authority references; +- export-policy version; +- revocation status when available. + +Passing verification means `cryptographically_valid`, not `locally_verified`. + +Community evidence must not: + +- append to `compatibility.jsonl`; +- rebuild or alter local canonical verdicts; +- refresh local evidence freshness; +- satisfy Routing Profile compatibility requirements; +- change Router Intelligence eligibility or scoring; +- trigger CL-08 refresh work; +- merge with local evidence into a single score. + +The UI/API/CLI must label it explicitly as community evidence and distinguish signature validity from compatibility truth. + +--- + +# 15. Community storage boundary + +Community bundles, if persisted, live outside the local canonical Lab ledger in a separate non-authoritative object/cache domain under the Lab root. + +Conceptually: + +```text +~/.opencodex/lab/ + compatibility.jsonl # local canonical authority, unchanged + compatibility.sqlite # local disposable projection, unchanged + artifacts/ # local Lab artifacts, unchanged + exports/ # user-created public bundles + community/ # non-authoritative imported public bundles/cache +``` + +The community store must not reuse local event IDs or masquerade as local observations. + +Deleting `community/` loses only imported community context and has no effect on local verdict reproducibility. + +--- + +# 16. Import boundary + +CL-10 V1 import accepts only bounded bundle bytes through reviewed entry points. It must not dereference arbitrary embedded URLs, paths, artifact references, or publisher-controlled network locations. + +A bundle is parsed with strict byte, UTF-8, duplicate-object-member, nesting, array, object-key, and string limits before expensive signature or projection work. Duplicate decoded object member names are rejected before `JSON.parse`-style semantic object construction so parsers cannot silently collapse an ambiguous wire representation. + +Invalid bundles are rejected without partial persistence. + +Artifact content embedded in/imported with a bundle is accepted only for closed `public_export` artifact classes and is revalidated locally before storage. + +--- + +# 17. Revocation and deletion semantics + +CL-10 defines `PublicEvidenceRevocationV1` as a signed, bounded public statement from the same publisher key that signed the target bundle and references one or more bundle/record IDs plus a finite reason code. + +A consumer bootstraps revocation authority from the already-verified target bundle: `publisher.keyId` and the exact Ed25519 public key in the revocation must match that target bundle before the revocation signature is considered. V1 does not support cross-key revocation or key rotation. A key-rotation protocol requires a later reviewed schema version. + +A revocation contains its own domain-separated digest/ID, `issuedDayUtc`, at most 256 sorted unique target IDs, and no free-form reason text. Re-importing the exact same revocation ID and bytes is idempotent. The same revocation ID with different canonical bytes, duplicate target IDs, an unknown target, unsupported reason/version, or a publisher-key mismatch is rejected. Consumers may retain bounded revocations received before a referenced record only in a quarantined pending set with the same structural limits; they do not become effective until the matching publisher/target bundle is present and verified. + +Allowed reason classes include: + +- `publisher_retracted`; +- `privacy_retraction`; +- `evidence_invalidated`; +- `superseded`. + +A revocation never edits the original local Lab ledger. + +Community consumers mark matching imported records revoked and exclude them from default community summaries while preserving the revocation audit relation. + +Remote physical deletion is a transport/service concern and cannot replace cryptographic revocation semantics. + +--- + +# 18. Remote publishing boundary + +This contract freezes bundle, consent, signing, verification, and trust semantics before choosing a remote service. + +No network publishing implementation is authorized until the same CL-10 branch or a reviewed follow-up contract records: + +- the exact service origin(s); +- authentication model, if any; +- maximum request/body budgets; +- TLS and redirect policy; +- retry/idempotency semantics; +- server retention and deletion policy; +- abuse/rate-limit behavior; +- revocation endpoint semantics; +- server-side schema validation; +- operator ownership and privacy policy. + +The publisher must not accept an arbitrary user-supplied upload URL as a shortcut around this gate. + +A fixed reviewed service may aggregate community bundles later, but local OpenCodex behavior remains fully functional without it. + +--- + +# 19. Read surfaces + +CL-10 implementation should extend existing Lab surfaces rather than create an unrelated product area. CL-10.1 through CL-10.4 are authorized after the accepted contract review; this does not relax the remote-publishing gate. + +Planned surfaces after contract acceptance: + +- CLI preview/export/verify/community inspection commands under `ocx lab`; +- authenticated management API for preview/export metadata and local community inspection; +- Compatibility Matrix detail UI for clearly separated community context; +- explicit publish UI only after the remote publishing transport contract is accepted. + +The local Compatibility Matrix must never silently replace its canonical verdict with a community result. + +--- + +# 20. Bounds + +V1 hard export/import ceilings: + +```text +maximum records per bundle 256 +maximum public artifacts per bundle 16 +maximum bytes per public artifact 256 KiB +maximum aggregate public artifact data 1 MiB +maximum serialized bundle bytes 2 MiB +maximum assertion summaries per record 64 +maximum incident references per record 32 +maximum serialized string field 4 KiB +maximum JSON nesting depth 8 +maximum object keys 64 +maximum array elements 512 +``` + +Implementations may use lower limits. Raising a hard ceiling requires a reviewed contract change. + +--- + +# 21. Failure semantics + +Export and import use explicit non-verdict outcomes. + +At minimum: + +```text +exportable +not_exportable +privacy_rejected +schema_rejected +signature_invalid +digest_invalid +revoked +unsupported_version +storage_failure +transport_unavailable +publish_rejected +``` + +These outcomes must never be mapped to local compatibility `DEGRADED` or `UNSUPPORTED` verdicts. + +--- + +# 22. Security tests required before implementation acceptance + +CL-10 implementation must include adversarial tests for: + +- prompt/response/tool/repository/path canaries; +- API keys, OAuth tokens, cookies, authorization headers, and common secret formats; +- account/email/project/tenant/location canaries; +- local subject/event/artifact/request/decision/Fabric IDs; +- custom provider/model IDs; +- URLs/query strings/IP addresses/header dumps; +- precise timestamps and raw latency/error fields; +- unknown JSON fields at every public schema level; +- duplicate decoded JSON object member names, including escape-equivalent keys; +- malformed/oversized/deeply nested import bundles; +- invalid signatures and digests; +- a fixed RFC 8785/domain-separated bundle digest and Ed25519 signature vector; +- bundle replay/deduplication; +- revoked bundles; +- community evidence isolation from local verdicts, routing, and CL-08; +- deterministic export from identical local inputs; +- non-exportability when private route dimensions would be erased. + +--- + +# 23. Delivery sequence + +## CL-10.0 - Audit and contract + +Contract work completed on this PR before runtime implementation: + +- record CL-09 closure; +- freeze public exportability and privacy rules; +- freeze public bundle schema and export-scoped identity; +- freeze publisher-signature and community trust semantics; +- freeze consent, revocation, import isolation, and remote-publishing gate; +- define implementation sequence and validation requirements. + +Independent review accepted CL-10.0 on 2026-08-12. CL-10.1 through CL-10.4 are therefore authorized on this branch by explicit maintainer direction. CL-10.5 remains blocked by section 18. + +## CL-10.1 - Public projector and privacy validator + +Implement closed public DTOs, exportability checks, export-scoped IDs, deterministic canonicalization, and fail-closed privacy validation. + +## CL-10.2 - Public bundle storage and publisher signatures + +Implement local public-bundle storage plus publisher-key lifecycle, bundle digesting, Ed25519 signing, and verification. + +## CL-10.3 - Local preview/export surfaces + +Implement CLI/API/UI preview and explicit local export. No remote publishing yet. + +## CL-10.4 - Community import and quarantine/read surfaces + +Implement strict import/verification, separate non-authoritative community storage, revocation handling, and clearly labelled read surfaces. No routing/verdict integration. + +## CL-10.5 - Remote publishing transport + +Implement only after the exact remote-service contract in section 18 is completed and independently accepted. + +## CL-10.6 - Adversarial closure and programme acceptance + +Run privacy, trust, cross-platform, no-feedback, reproducibility, and independent review gates. On acceptance, mark Compatibility Lab CL-00 through CL-10 complete. + +--- + +# 24. Explicit non-goals + +CL-10 V1 must not implement: + +- automatic telemetry; +- background production evidence upload; +- raw local Lab ledger export; +- custom/private route publication; +- user prompt/response/tool/repository export; +- account-linked public identity; +- community evidence as canonical local evidence; +- community-driven Routing Profile or Router Intelligence behavior; +- community-driven CL-08 scheduling; +- global compatibility score or leaderboard that mixes incomparable evidence layers; +- arbitrary upload/download URLs; +- remote code/tool execution; +- public artifact classes without explicit `public_export` policy. + +--- + +# 25. Contract acceptance criteria + +CL-10.0 is accepted only when independent review agrees that: + +1. no local/private identifier is required by the public schema; +2. exact local evidence cannot be generalized into a misleading public claim by dropping private route dimensions; +3. exported fields are closed, bounded, versioned, and fail closed on unknown fields; +4. public artifacts require explicit opt-in policy and second-pass sanitization; +5. export/publish requires explicit user action and creates no automatic telemetry path; +6. publisher signatures prove integrity/continuity without being misrepresented as evidence truth; +7. imported community evidence is isolated from local canonical evidence, freshness, routing, and scheduling; +8. revocation semantics are defined independently of remote physical deletion; +9. remote transport remains gated until an exact service/security contract exists; +10. implementation tasks have adversarial privacy and trust tests sufficient to prevent silent boundary regression. + +--- + +# 26. Validation + +Contract PR minimum: + +```text +git diff --check +repository markdown / hygiene checks +CodeRabbit / independent review +``` + +Implementation phases must additionally run: + +```text +bun x tsc --noEmit +bun run privacy:scan +focused Lab export/import/signature tests +focused ledger/projection isolation tests +Routing Profile / Router Intelligence no-feedback regressions +CL-08 no-feedback regressions +CLI/API/GUI tests for implemented surfaces +cross-platform CI +``` + +--- + +# 27. Hard stop + +The CL-10 contract was independently accepted on 2026-08-12 and explicit maintainer direction authorizes CL-10.1 through CL-10.4 runtime implementation on this branch. + +No CL-10.5 remote publishing code, upload transport, remote fetch, or arbitrary network publication is authorized until section 18 has been completed with an exact reviewed transport contract and independently accepted. diff --git a/devlog/_plan/260807_compatibility_lab/011_cl10_revocation_v1_anchor_clarification.md b/devlog/_plan/260807_compatibility_lab/011_cl10_revocation_v1_anchor_clarification.md new file mode 100644 index 0000000000..829e8aefae --- /dev/null +++ b/devlog/_plan/260807_compatibility_lab/011_cl10_revocation_v1_anchor_clarification.md @@ -0,0 +1,17 @@ +# CL-10 V1 revocation anchor clarification + +Status: normative clarification to `010_cl10_public_evidence_export.md` section 17. + +`PublicEvidenceRevocationV1` uses exactly one already-verified target bundle as its authority anchor. The revocation may contain between 1 and 256 sorted unique targets, but every target must resolve inside that one anchor bundle: + +- a `bundle` target must equal the anchor bundle ID; +- a `record` target must name a record contained by the anchor bundle; +- mixed bundle/record targets are allowed only when they all resolve inside the same anchor bundle; +- multiple distinct bundle IDs in one V1 revocation are not supported and must be rejected; +- targets spread across multiple bundles are not supported even when those bundles use the same publisher key. + +The publisher algorithm, key ID and exact public key in the revocation must match the already-verified anchor bundle before the revocation signature is authoritative. V1 therefore has no cross-key, key-rotation or multi-bundle authority bootstrap. + +The phrase "one or more bundle/record IDs" in section 17 describes the bounded target list, not multiple independent bundle authority contexts. Where that wording could be read as authorizing a single V1 revocation across multiple bundles, this clarification is authoritative. + +Supporting multi-bundle revocation requires a separately reviewed contract/schema revision that defines how all target bundles are supplied, verified, bounded and bound to the signing authority before persistence or application. diff --git a/devlog/_plan/260815_open_pr_triage/000_plan.md b/devlog/_plan/260815_open_pr_triage/000_plan.md new file mode 100644 index 0000000000..751cbe7fd3 --- /dev/null +++ b/devlog/_plan/260815_open_pr_triage/000_plan.md @@ -0,0 +1,45 @@ +# 000 — 260815_open_pr_triage: Plan + +## Objective + +Triage every open PR shown in the owner's 2026-08-15 list (22 PRs, #1704-#1732) in lidge-jun/opencodex. Each PR gets exactly one executed disposition: MERGE (squash via gh, or cherry-pick when unmergeable), CLOSE with evidence-led comment, or KEEP-DRAFT with named gaps. Owner directives: suites run only on ssh lidge; pushing to dev is authorized (--no-verify allowed); unlimited subagents; multi-cycle PABCD. + +## Loop-spec + +- Loop archetype: verifier-defined (gh PR state + lidge suite green on dev). +- Write scope: GitHub PR state (merge/close/comment), devlog unit docs. Out-of-scope: main/preview branches, npm release, issues, PRs #1703 and older (not in the owner's list). +- Budget / bounds: wall-clock one session; BLOCKED if lidge or gh auth fails. + +## Evidence base + +- 5 explorer subagent verdicts (diff-level per-PR review), 2026-08-15. +- GraphQL reviewThreads (authority for unresolved blocking threads). +- Exact-head check-run rollup per PR (gh pr view --json statusCheckRollup). + +## Work-phase map (one phase = one full PABCD cycle) + +| WP | Doc | Slice | Depends on | +|----|-----|-------|------------| +| wp1 | 010 | Triage matrix (this unit's docs) | - | +| wp2 | 020 | Lab CL merges: #1708 #1709 #1710 #1712 #1715 #1717 #1719 #1720, then #1705 -> #1706 stack | wp1 | +| wp3 | 020 | Adapters: #1714 merge, then #1721 rebase + fixture fix + merge; #1722/#1723 stay draft (Major thread) | wp2 | +| wp4 | 020 | Contributor #1716 merge; KEEP-DRAFT comments #1718 #1725 #1728; #1704 stays draft | wp2 | +| wp5 | 020 | Storage/log-guard #1727 #1729 #1732 stay draft (named gaps) | wp1 | +| wp6 | 030 | Final: lidge full suite on dev, verify all dispositions, report | wp2-wp5 | + +## Merge mechanics (B-phase execution) + +1. Independent lab fixes first (disjoint files; overlaps #1709/#1717 and #1712/#1720 verified clean by merge-tree): gh pr merge --squash (--admin only if the review-requirement blocks; owner-directed triage). +2. #1705 merges; GitHub auto-retargets #1706 to dev; merge #1706. +3. #1714 merges. Then #1721: rebase onto dev, switch mimo-free fixture URL to canonical MIMO_CHAT_URL (semantic conflict with #1714's endpoint guard), push, merge. +4. #1716 merges (external contributor - use gh merge so it records as merged). +5. KEEP-DRAFT set: no state change; gaps recorded in 010 matrix; brief maintainer comment on external drafts (#1718 #1725 #1728). + +## Accept criteria + +- c1: 010 matrix written with per-PR evidence (this unit). +- c2: every listed PR shows merged/closed/draft disposition via gh pr view. +- c3: full suite green on lidge against final dev head. +- c4: no security notes in tracked files (log-guard findings stay in matrix form - all referenced fixes already public in PR diffs). +- c5: final per-PR outcome report. + diff --git a/devlog/_plan/260815_open_pr_triage/010_triage_matrix.md b/devlog/_plan/260815_open_pr_triage/010_triage_matrix.md new file mode 100644 index 0000000000..8d6289e3a5 --- /dev/null +++ b/devlog/_plan/260815_open_pr_triage/010_triage_matrix.md @@ -0,0 +1,51 @@ +# 010 - Triage matrix (wp1 deliverable) + +Evidence gathered 2026-08-15. "checks" = exact-head check-runs; "threads" = fresh unresolved reviewThreads (GraphQL). Verdicts from 5 independent explorer subagents, spot-verified by maintainer agent. + +A-audit amendment (GO-WITH-FIXES, 2 blockers folded): +1. #1706 will be retargeted to dev explicitly ('gh pr edit 1706 --base dev') before merging - the repo does not auto-delete branches, so merge does not auto-retarget stacked children. +2. #1722/#1723 upgraded KEEP-DRAFT -> MERGE: the Cursor false-pass Major was fixed at head ca0a5124 (force-push 02:13Z, after the 01:56Z CodeRabbit comment); same fixed-in-branch standard the plan already applied to #1706. Residual: stale-thread hygiene only. + +## MERGE (15) + +| PR | What | Gate state | Notes | +|----|------|-----------|-------| +| #1708 | fix(lab) CL-01: negative control asserted its own repair | 35 green, 0 threads | harness integrity | +| #1709 | fix(lab) CL-02: transactional ledger mutations | 26 green, 0 threads | TOCTOU races closed | +| #1710 | fix(lab) CL-03: distinct transport failure classes | 35 green, 1 outdated | backward compatible | +| #1712 | fix(lab) CL-04: 400 on invalid read filters | 35 green, 1 minor | empty-artifactClass follow-up optional | +| #1715 | fix(lab) CL-05: GUI partial-read failures | 29 green, 0 threads | gui-screenshot-waived label present | +| #1717 | fix(lab) CL-07: fail-closed outcome validation | 29 green, 0 threads | clean vs #1709 | +| #1719 | fix(lab) CL-08: rebind runtime ownership on server replace | 32 green, 0 threads | lifecycle defect | +| #1720 | fix(lab) CL-09: passive read surface alignment | 32 green, 0 threads | clean vs #1712 | +| #1705 | feat(lab) CL-10 trust core | 31 green (react-doctor cancelled = superseded dup), 2 trivial + 1 minor threads | stack base | +| #1706 | feat(lab) CL-10 operator/community | 26 green, Major fixed in-branch (7a1e066ae) | explicit retarget to dev before merge; core-lab boundary test green | +| #1714 | fix(providers) static model discovery | 32 green, 0 threads | land BEFORE adapter stack (endpoint-guard fixture conflict) | +| #1721 | refactor(adapters) registry authority Part 1 | 26 green, 0 threads | rebase + mimo fixture fix (authority test only) after #1714 | +| #1722 | test(adapters) conformance Part 2 | 23 green; Major fixed at head ca0a5124 | stack order after #1721; A-audit upgrade | +| #1723 | test(adapters) buffered freeform Part 3 | 23 green, 0 threads | stack order after #1722; A-audit upgrade | +| #1716 | feat(models) per-custom-model reasoning effort | 27 green (+1 cancelled react-doctor dup; win shard skip systemic), screenshot present | "Critical duplicate payload" verified false positive; 1 minor trim nit | + +## KEEP-DRAFT (7) + +| PR | Why not now | Gap to merge | +|----|-------------|--------------| +| #1704 | own GUI quota PR: hygiene + enforce-target FAIL (no screenshot, no regression test), 5 fresh threads | screenshot, unit test, address threads | +| #1718 | external draft, light gates only, no full CI | author checklist + maintainer-triggered CI | +| #1725 | external draft, no full CI | author readiness + maintainer CI | +| #1727 | enforce-target (screenshot), hygiene empty_catch, unresolved Major: sqliteHome/databasePath leak in API/CLI/UI | redact path fields, fixed-message errors, screenshot | +| #1728 | author mid-flight: checklist 0/4, manual verification pending | author completes checklist | +| #1729 | 5 real macOS test failures in protection/lock core, 10 unresolved threads | fix lock/trigger semantics failures | +| #1732 | failing reclaim/gates checks, plausible TOCTOU + stopReason Majors, CI still running | post-open path re-validation, stopReason fix, drop duplicate workflow | + +## Stacks and order constraints + +- #1705 (base dev) -> #1706 (base cl10-public-core): merge #1705, then 'gh pr edit 1706 --base dev', then merge #1706. +- #1721 -> #1722 -> #1723 stack: merge in order; use --delete-branch on each merge so the next child retargets to dev automatically (or retarget explicitly). +- #1727 -> #1729 -> #1732 stack: all stay draft. +- Semantic conflict: #1714's canonical-endpoint guard breaks the mimo-free fixture in #1721's authority test (example.invalid/v1). #1714 lands first; #1721 rebases with MIMO_CHAT_URL fixture before merge. #1722/#1723 fixtures re-verified against the combined tree before their merges. + +## Out of scope (open but not in owner's list) + +#1703 #1669 #1664 #1660 #1655 #1652 #1645 #1644 #1624 #1584 #1569 #1557 #1552 #1526 #1521 #1498 #1367 #1165 - untouched this round. + diff --git a/devlog/_plan/260815_open_pr_triage/020_merge_execution.md b/devlog/_plan/260815_open_pr_triage/020_merge_execution.md new file mode 100644 index 0000000000..4ac3dd12c0 --- /dev/null +++ b/devlog/_plan/260815_open_pr_triage/020_merge_execution.md @@ -0,0 +1,65 @@ +# 020 - Merge execution record (wp2-wp3) + +## Plan (written at P, verified: zero head drift on all 15 PRs, 2026-08-15) + +Mechanism: local integration branch with --no-ff merges of pull//head refs, +so each PR head becomes an ancestor of dev and GitHub auto-marks the PR Merged +on push (preserves contributor attribution, e.g. external #1716). One dev push, +one dev CI run, one lidge validation before the push. + +### Step 1 - retarget stacked children to dev (gh pr edit --base dev) + +- #1706 (was cl10-public-core), #1722 (was refactor/adapter-registry-authority), + #1723 (was test/adapter-registry-conformance). + +### Step 2 - integration branch + +git fetch origin +git switch -c int/260815-pr-landings origin/dev + +Merge order (dependency-safe): +1. #1708 #1709 #1710 #1712 #1715 #1717 #1719 #1720 (independent lab fixes) +2. #1705 then #1706 (stack; #1706 branch contains #1705) +3. #1714 (endpoint guard) +4. #1721 (authority test fixture fix required: tests/adapter-registry-authority.test.ts + mimo-free provider baseUrl example.invalid/v1 -> canonical MIMO_CHAT_URL + https://api.xiaomimimo.com/api/free-ai/openai/chat; separate fix commit) +5. #1722 then #1723 (stack; contains #1721) +6. #1716 (external feature, disjoint files) + +Each: git merge --no-ff FETCH_HEAD -m 'Merge PR #: ' using +git fetch origin pull/<n>/head. + +### Step 3 - devlog unit onto int + +Cherry-pick 1628d06c2 (triage docs) onto int. + +### Step 4 - validate + +git push origin int/260815-pr-landings +ssh lidge: clone/fetch, checkout int branch, bun install, bun run typecheck + +bun run test (+ privacy:scan). Suite runs ONLY on lidge per owner directive. + +### Step 5 - land + +git push origin int/260815-pr-landings:dev --no-verify +(owner-authorized; enforce_admins=false so admin bypass works on protected dev) +Then verify all 15 PRs auto-marked Merged; stragglers get an evidence comment +and manual close. + +### Step 6 (wp3) - KEEP-DRAFT comments + +Brief maintainer comment on #1704 #1718 #1725 #1727 #1728 #1729 #1732 naming +the recorded gaps (010 matrix). + +## A-audit amendments (GO-WITH-FIXES, 3 blockers folded) + +1. #1709 -> #1706 semantic conflict in src/lab/ledger/purge.ts: #1706's export-purge steps + deferred-error vars must be re-expressed inside #1709's withLedgerMutation wrapper. Pre-staged resolution; purge tests re-run. +2. #1715 -> #1714 trivial conflict in gui/.eslint/i18n-allowlist.ts: take #1714's /^HTTP$/i version (superset). +3. Fixture fix covers THREE files (mimo-free canonical /chat under #1714's guard): tests/adapter-registry-authority.test.ts (#1721), tests/adapter-tool-conformance.test.ts (#1722), tests/adapter-buffered-tool-conformance.test.ts (#1723). Separate commits, never amend PR heads (auto-merge detection is exact-SHA). +4. Advisory: retargets before push; re-verify all 15 head SHAs at push time. + +## Execution log + +(pending) + diff --git a/devlog/_plan/260815_open_pr_triage/030_final_verification.md b/devlog/_plan/260815_open_pr_triage/030_final_verification.md new file mode 100644 index 0000000000..44659ec5c6 --- /dev/null +++ b/devlog/_plan/260815_open_pr_triage/030_final_verification.md @@ -0,0 +1,4 @@ +# 030 - Final verification (wp6) + +(pending: lidge suite output on final dev head, gh disposition verification, per-PR outcome report) + diff --git a/docs-site/src/content/docs/guides/pi.md b/docs-site/src/content/docs/guides/pi.md index fa44d2754f..c44b97f12a 100644 --- a/docs-site/src/content/docs/guides/pi.md +++ b/docs-site/src/content/docs/guides/pi.md @@ -103,9 +103,26 @@ small-context model is never given more output than context. It is not a claim a model's true maximum. Two fields are deliberately absent. `cost` requires all four price fields and opencodex has no -price data for routed models — emitting zeros would assert that every model is free. `reasoning` is -a boolean in Pi while the catalog carries an effort ladder, and mapping one onto the other would be -a guess. +price data for routed models — emitting zeros would assert that every model is free. + +`reasoning` is the one field that used to be absent and now is not: Pi stores a boolean while the +catalog carries an effort ladder, and mapping one onto the other used to be a guess. Since the +catalog's ladder is the proxy's own statement about whether a model accepts reasoning parameters +(adapters honor `reasoning_effort`), an export row with a **non-empty** ladder now emits +`"reasoning": true`, and a row without one (or with an explicitly empty ladder) stays +reasoning-free. Pi then offers its effort control for exactly the models opencodex will accept it +on. The export also emits a `thinkingLevelMap` that hides every pi level with no declared target +(`null`), so pi never offers — and never sends — an effort the ladder does not contain. One +fallback keeps the model usable: when `ultra` is declared without `max`, pi's `max` level maps +to `ultra` (still a ladder member). +If you need a different mapping, hand-edit `thinkingLevelMap` afterward as documented by Pi. + +Treat `reasoning` as Pi-UI metadata: it is derived from the catalog ladder, not proof that the +upstream natively supports a reasoning parameter. What the proxy actually sends for a given +`reasoning_effort` value depends on the provider's adapter and model — it may pass the value +through, translate it (wire aliases), clamp it to the configured ladder, emulate it, or omit it +entirely (e.g. `noReasoningModels`). The boolean only controls whether Pi offers the control at +all. ## Schema status diff --git a/docs/screenshots/custom-model-reasoning-dialog.png b/docs/screenshots/custom-model-reasoning-dialog.png new file mode 100644 index 0000000000..633c2159ae Binary files /dev/null and b/docs/screenshots/custom-model-reasoning-dialog.png differ diff --git a/docs/superpowers/plans/2026-08-12-cl10-public-evidence-implementation.md b/docs/superpowers/plans/2026-08-12-cl10-public-evidence-implementation.md new file mode 100644 index 0000000000..5c58021baa --- /dev/null +++ b/docs/superpowers/plans/2026-08-12-cl10-public-evidence-implementation.md @@ -0,0 +1,156 @@ +# CL-10 Public Evidence Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Implement CL-10.1 through CL-10.4: deterministic privacy-safe public evidence projection, signed local bundles, explicit local export, and quarantined community import/read surfaces, while keeping remote publishing blocked. + +**Architecture:** Add a dedicated `src/lab/public/` boundary with independently versioned public types and strict validators. Public bundles are derived from valid local Lab evidence only after an exact exportability gate, signed with a local Ed25519 publisher key, and stored separately from the canonical ledger. Imported bundles are bounded, signature-checked, and stored only in a non-authoritative community domain that never feeds local verdicts, routing, or CL-08. + +**Tech Stack:** TypeScript, Bun tests, Node `crypto` Ed25519, existing Lab JSONL/SQLite/query/digest/path infrastructure, existing `ocx lab` CLI and authenticated management API, existing Compatibility Matrix UI/i18n. + +## Global Constraints + +- No automatic telemetry or background publishing. +- No remote publishing implementation in this plan; CL-10.5 remains blocked until an exact reviewed service contract exists. +- No local subject/event/artifact/request/decision/Fabric identifier may appear in a public bundle. +- Private/custom route dimensions make evidence `not_exportable`; they are never dropped to broaden a public claim. +- Public schemas are closed and independently versioned; unknown fields fail closed. +- Public route identity uses a repo-reviewed, versioned allowlist authority. Dynamic discovery/configuration cannot extend it. +- Public incident references are closed corpus IDs only; historical URLs/devlog paths are never exported. +- Community evidence is `community_untrusted_v1`, never canonical local evidence, freshness, routing, or CL-08 input. +- Sensitive purge removes affected generated exports and locally-originated community copies; network revocation is never a prerequisite for completing a local purge. +- Publisher signatures prove integrity/continuity only, not evidence truth. + +--- + +### Task 1: Freeze review amendments and implementation authority + +**Files:** +- Modify: `devlog/_plan/260807_compatibility_lab/010_cl10_public_evidence_export.md` +- Modify: `docs/superpowers/specs/2026-08-12-cl10-public-evidence-design.md` + +**Interfaces:** +- Consumes: CL-00 purge/public-export contracts and merged CL-09 state. +- Produces: final CL-10.1–CL-10.4 runtime contract; CL-10.5 remains explicitly blocked. + +- [ ] **Step 1:** Add explicit purge/export/community-copy semantics consistent with CL-00 `purgeActions: export`. +- [ ] **Step 2:** Define `PublicRouteRegistryManifestV1` as the versioned local trust anchor for public provider/model identity. +- [ ] **Step 3:** Define bounded revocation bootstrap: target publisher key must match the original bundle publisher; duplicates are idempotent; conflicting replay fails closed; no V1 key rotation. +- [ ] **Step 4:** Replace arbitrary `incidentRefs` with closed `IC-NNN` references and require `artifactRefs` to resolve only to public artifact IDs in the same bundle. +- [ ] **Step 5:** Replace the route-only record assumption with a closed `PublicEvidenceSubjectV1` union for protocol/route/task evidence and require dedicated runtime validators/types. +- [ ] **Step 6:** Record that independent review accepted the contract and the user authorized CL-10.1–CL-10.4 runtime implementation on this PR; preserve the CL-10.5 transport hard stop. + +### Task 2: Public schema, registry authority, and privacy projector + +**Files:** +- Create: `src/lab/public/types.ts` +- Create: `src/lab/public/registry.ts` +- Create: `src/lab/public/validate.ts` +- Create: `src/lab/public/project.ts` +- Create: `src/lab/public/index.ts` +- Modify: `src/lab/index.ts` +- Test: `tests/lab-public-evidence.test.ts` + +**Interfaces:** +- Produces: `PublicEvidenceBundleUnsignedV1`, `PublicEvidenceRecordV1`, `PublicEvidenceSubjectV1`, `PublicRouteRegistryManifestV1`, `projectPublicEvidence()`, `validatePublicEvidenceBundle()`. + +- [ ] **Step 1: Write RED tests** for closed-schema rejection, deterministic public IDs/day buckets, protocol/route/task subject discrimination, exact route allowlist, private-route `not_exportable`, IC-only incident refs, no local ID leakage, and secret/PII canaries. +- [ ] **Step 2: Run focused test and verify expected RED failures.** + Run: `bun test tests/lab-public-evidence.test.ts` +- [ ] **Step 3: Implement minimal closed public types/registry/validator/projector.** + Public identities use domain-separated SHA-256 over JCS public-safe bytes. The registry manifest is repo-owned, versioned, digested, and cannot be supplied by an imported bundle as trust authority. +- [ ] **Step 4: Run focused test and verify GREEN.** + +### Task 3: Bundle digest/signature and local storage + +**Files:** +- Create: `src/lab/public/signature.ts` +- Create: `src/lab/public/storage.ts` +- Modify: `src/lab/paths.ts` +- Test: `tests/lab-public-evidence.test.ts` + +**Interfaces:** +- Produces: `getOrCreatePublicPublisher()`, `signPublicEvidenceBundle()`, `verifyPublicEvidenceBundle()`, `writePublicEvidenceBundle()`, `readPublicEvidenceBundle()`. + +- [ ] **Step 1: Write RED tests** for Ed25519 signing/verification, key-file permissions where enforceable, tamper rejection, deterministic bundle digest, bounded storage paths, and no private-key serialization. +- [ ] **Step 2: Verify RED.** +- [ ] **Step 3: Implement minimal key lifecycle, signing, verification, and safe local bundle storage.** +- [ ] **Step 4: Verify GREEN.** + +### Task 4: Revocation and community quarantine + +**Files:** +- Create: `src/lab/public/revocation.ts` +- Create: `src/lab/public/community.ts` +- Test: `tests/lab-public-evidence.test.ts` + +**Interfaces:** +- Produces: `PublicEvidenceRevocationV1`, `verifyPublicEvidenceRevocation()`, `importCommunityBundle()`, `listCommunityBundles()`. + +- [ ] **Step 1: Write RED tests** proving revocation accepts only the original bundle publisher key, duplicate identical revocations are idempotent, conflicting replay rejects, malformed/oversized bundles reject before persistence, and community import leaves canonical JSONL/SQLite verdict state unchanged. +- [ ] **Step 2: Verify RED.** +- [ ] **Step 3: Implement bounded revocation verification and separate community storage.** +- [ ] **Step 4: Verify GREEN.** + +### Task 5: Sensitive purge integration + +**Files:** +- Modify: `src/lab/ledger/purge.ts` +- Modify: `src/lab/paths.ts` +- Test: `tests/lab-public-evidence.test.ts` +- Test: `tests/lab-evidence-ledger.test.ts` + +**Interfaces:** +- Consumes: existing `purgeSensitiveEvidence()` and `purgeActions: export`. +- Produces: fail-closed removal of generated exports and locally-originated community copies affected by local sensitive evidence. + +- [ ] **Step 1: Write RED purge regression** showing an `export` purge removes CL-10 exports and local-origin community copies without requiring network access. +- [ ] **Step 2: Verify RED.** +- [ ] **Step 3: Extend purge-owned local directories/metadata minimally.** +- [ ] **Step 4: Run CL-10 and existing ledger purge tests.** + +### Task 6: Explicit CLI and management surfaces + +**Files:** +- Modify: `src/cli/lab.ts` +- Modify: `src/server/management/lab-routes.ts` +- Test: `tests/lab-public-evidence.test.ts` +- Test: relevant Lab CLI/management tests discovered in repository. + +**Interfaces:** +- CLI: local preview/export, bundle verify, community import/list. No publish command. +- API: authenticated preview/export/verify/community endpoints only. No remote transport. + +- [ ] **Step 1: Write RED CLI/API tests** for network-free preview, explicit export, verification, bounded community import, and absence of any publish endpoint/command. +- [ ] **Step 2: Verify RED.** +- [ ] **Step 3: Implement minimal surfaces using the public module APIs.** +- [ ] **Step 4: Verify focused CLI/API tests GREEN.** + +### Task 7: Compatibility Matrix community context + +**Files:** +- Modify: `gui/src/pages/compatibility-matrix-api.ts` +- Modify: `gui/src/pages/CompatibilityMatrix.tsx` +- Modify: locale catalog files under `gui/src/i18n/` as required by existing i18n rules. +- Test: existing Compatibility Lab GUI/i18n tests plus focused CL-10 additions. + +**Interfaces:** +- Produces: clearly labelled, read-only community context separate from canonical local verdict UI. + +- [ ] **Step 1: Write RED parser/render/i18n tests** proving community state is labelled non-authoritative and cannot replace the local verdict. +- [ ] **Step 2: Verify RED.** +- [ ] **Step 3: Implement the compact existing-detail-pane integration with no new product area.** +- [ ] **Step 4: Run GUI tests/lint/build GREEN.** + +### Task 8: Closure validation + +**Files:** +- Modify docs only if validation findings require factual updates. + +- [ ] **Step 1:** Run `bun test tests/lab-public-evidence.test.ts tests/lab-evidence-ledger.test.ts`. +- [ ] **Step 2:** Run `bun x tsc --noEmit`. +- [ ] **Step 3:** Run `bun run privacy:scan`. +- [ ] **Step 4:** Run relevant Lab query/ledger/CLI/GUI tests. +- [ ] **Step 5:** Run GUI lint/build and React Doctor. +- [ ] **Step 6:** Run full Cross-platform CI on the exact final PR head. +- [ ] **Step 7:** Confirm no remote publishing code, arbitrary URL transport, routing feedback, local-verdict feedback, or CL-08 feedback was introduced. diff --git a/docs/superpowers/plans/2026-08-13-cl10-deep-review-hardening.md b/docs/superpowers/plans/2026-08-13-cl10-deep-review-hardening.md new file mode 100644 index 0000000000..e1ccfccbcf --- /dev/null +++ b/docs/superpowers/plans/2026-08-13-cl10-deep-review-hardening.md @@ -0,0 +1,200 @@ +# CL-10 Deep Review Hardening Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Close all twelve adversarial findings from the post-CI CL-10 deep review, remove the catalog-timeout workaround, and make PR #1510 accurately describe the implemented CL-10.1 through CL-10.4 runtime scope. + +**Architecture:** Keep the existing `src/lab/public/` trust boundary and wire schema, but make verification canonical instead of normalizing attacker input, make community import no more permissive than local export, make revocation application publisher-scoped, and use crash-safe immutable-file publication. Public API/CLI DTOs remain separate from local operator metadata, and purge gains a bounded public-origin index so provenance does not depend on recovering mutable local files. + +**Tech Stack:** TypeScript, Bun tests, Node `crypto`/`fs`, existing Lab JCS/digest/path infrastructure, GitHub Actions. + +## Global Constraints + +- CL-10.5 remote publishing remains blocked and must not be implemented. +- No automatic telemetry, background publishing, arbitrary URL fetch, or community-to-local authority feedback. +- Keep `PublicEvidenceBundleV1` and revocation V1 domain strings frozen. +- New production behavior must be introduced test-first. +- Public bundle verification must reject non-canonical wire order rather than silently normalize it. +- Until a reviewed `public_export` artifact authority exists, both local and community V1 paths reject non-empty public artifacts. +- Publisher-key creation must not occur for invalid signing or revocation requests. +- Public management/CLI JSON must not disclose local filesystem paths or local Lab event IDs. +- Sensitive purge must remove locally-originated community copies even if the export or publisher key is damaged or missing. +- Exact-head GitHub Actions success is required before completion; do not merge. + +--- + +### Task 1: Add adversarial RED coverage + +**Files:** +- Create: `tests/lab-public-deep-review-regressions.test.ts` +- Modify: `tests/ci-workflows.test.ts` + +**Interfaces:** +- Consumes: current CL-10 public module APIs. +- Produces: failing tests for canonical array order, artifact quarantine, publisher-scoped record revocation, invalid-input key non-creation, JCS Unicode validity, exact assertion authority, cache quota, public DTO redaction, purge origin recovery, bounded duplicate-key diagnostics, IPv6 privacy rejection, and test-local catalog timeout behavior. + +- [ ] **Step 1:** Add one focused regression per finding using real public module behavior and deterministic test-only publisher keys where signatures are required. +- [ ] **Step 2:** Add a CI-policy regression that requires the catalog hardening test to own its timeout and forbids a catalog-specific timeout branch in the Linux batch runner. +- [ ] **Step 3:** Push tests only and verify the exact test-only head is red for the intended missing behavior. + +### Task 2: Canonical wire verification and JCS correctness + +**Files:** +- Modify: `src/lab/conformance/jcs.ts` +- Modify: `src/lab/public/bundle.ts` +- Modify: `src/lab/public/signature.ts` + +**Interfaces:** +- Produces: strict RFC-8785-compatible Unicode rejection and `verifyPublicEvidenceBundle()` rejection of non-canonical top-level record/artifact order. + +- [ ] **Step 1:** Reject lone UTF-16 surrogates in JCS strings and object keys. +- [ ] **Step 2:** Normalize bundle content once for local construction, but compare received record/artifact ordering against that normalized representation during verification. +- [ ] **Step 3:** Run the focused wire regressions green. + +### Task 3: Align community artifact/privacy authority + +**Files:** +- Modify: `src/lab/public/community.ts` + +**Interfaces:** +- Consumes: `validatePublicEvidencePrivacy()` and the current V1 artifact hard stop. +- Produces: community imports that reject all non-empty artifacts until reviewed authority exists and run the same second-pass privacy validator before persistence. + +- [ ] **Step 1:** Add a community-import gate before persistence. +- [ ] **Step 2:** Verify signed artifact-bearing external bundles are rejected and artifact-empty valid bundles still import. + +### Task 4: Make record revocation publisher-scoped + +**Files:** +- Modify: `src/lab/public/community.ts` +- Modify: `src/lab/public/revocation.ts` only if helper semantics need to be exposed. + +**Interfaces:** +- Produces: deterministic verification of record-only revocations against any matching verified bundle for the publisher and application to every matching record in that publisher's imported bundles. + +- [ ] **Step 1:** Resolve record-only revocation authority against a deterministic matching verified bundle instead of requiring exactly one bundle. +- [ ] **Step 2:** During listing, apply each verified revocation by publisher plus bundle/record target membership rather than binding it permanently to one bundle. +- [ ] **Step 3:** Verify a later bundle containing the same record remains revoked. + +### Task 5: Crash-safe immutable persistence and key lifecycle + +**Files:** +- Create: `src/lab/public/private-file.ts` +- Modify: `src/lab/public/signature.ts` +- Modify: `src/lab/public/storage.ts` +- Modify: `src/lab/public/community.ts` + +**Interfaces:** +- Produces: temp-file + file fsync + exclusive hard-link publication for immutable secret/public objects, deterministic EEXIST conflict handling, POSIX parent-directory durability before success is reported, an explicit Windows fallback where directory fsync is not portable, and test-only publication fault seams. + +- [ ] **Step 1:** Implement a small shared helper that writes a mode-0600 private temp file, fsyncs it, publishes it by exclusive hard link, fsyncs the parent directory on POSIX, and removes the temp name only after the publication durability boundary succeeds. On Windows, retain atomic exclusive publication without requiring unsupported directory fsync. +- [ ] **Step 2:** Migrate publisher-key creation, local exports, and community bundle/revocation persistence to the helper. +- [ ] **Step 3:** Verify an injected pre-publish failure leaves no final partial file, a POSIX parent-directory-sync failure is reported and can be recovered by an idempotent retry, and Windows publication does not depend on directory fsync. + +### Task 6: Validate before publisher-state mutation + +**Files:** +- Modify: `src/lab/public/bundle.ts` +- Modify: `src/lab/public/signature.ts` +- Modify: `src/lab/public/revocation.ts` + +**Interfaces:** +- Produces: a publisher-independent content-normalization function used before key access; revocation creation requires an existing matching local publisher key. + +- [ ] **Step 1:** Split public bundle content validation/normalization from publisher attachment. +- [ ] **Step 2:** Run closed-schema/day/record/authority/privacy validation before `getOrCreatePublicPublisher()`. +- [ ] **Step 3:** Add an existing-publisher loader and use it for revocation creation so foreign/invalid revocation attempts cannot create identity state. + +### Task 7: Exact assertion authority + +**Files:** +- Modify: `src/lab/public/community-authority.ts` + +**Interfaces:** +- Produces: exact one-to-one assertion-ID/required-flag coverage of the reviewed scenario authority. + +- [ ] **Step 1:** Reject duplicate assertion IDs. +- [ ] **Step 2:** Reject missing reviewed assertions as well as unknown ones. +- [ ] **Step 3:** Keep passed/failed values publisher-supplied evidence while freezing only identity/required authority. + +### Task 8: Bound community cache writes and read cost + +**Files:** +- Modify: `src/lab/public/community.ts` + +**Interfaces:** +- Produces: pre-create limits of 512 cache files and 64 MiB aggregate serialized bytes, with idempotent existing objects still readable/importable at the limit. + +- [ ] **Step 1:** Measure only descriptor-bound regular files without following symlinks. +- [ ] **Step 2:** Enforce count and aggregate-byte capacity before creating a new object. +- [ ] **Step 3:** Enforce the same bounds when listing so corrupted/external directory growth fails closed before bulk materialization. + +### Task 9: Separate public DTOs from local operator metadata + +**Files:** +- Modify: `src/lab/public/operator.ts` +- Modify: `src/cli/lab.ts` +- Modify: `src/server/management/lab-routes.ts` +- Modify: `tests/lab-public-surfaces.test.ts` + +**Interfaces:** +- Produces: preview/export results that expose exclusion indices/reasons and `stored.created` only, never local event IDs or filesystem paths. + +- [ ] **Step 1:** Replace public exclusion `eventId` with bounded `selectionIndex`. +- [ ] **Step 2:** Discard storage paths from public operator return values and CLI/API JSON. +- [ ] **Step 3:** Keep human CLI output useful without printing local absolute paths. + +### Task 10: Persist public-origin provenance for purge + +**Files:** +- Modify: `src/lab/paths.ts` +- Create: `src/lab/public/origin.ts` +- Modify: `src/lab/public/operator.ts` +- Modify: `src/lab/public/purge.ts` + +**Interfaces:** +- Produces: bounded immutable `public-origin-v1` markers containing only public publisherKeyId/bundleId identities. The origin marker is durably committed before a new local export file is published, so export success can never be reported without purge-owned provenance; an orphan marker after a later export failure is conservative and safe. Under retention pressure, markers without an exact community bundle copy may be reclaimed because no community object remains for that provenance marker to classify; markers backing retained community bundles are preserved. + +- [ ] **Step 1:** Commit the public origin identity before publishing the local export file; if export publication later fails, preserve the orphan marker so retry/purge can recover conservatively. +- [ ] **Step 2:** Make purge union origin markers with legacy recoverable export/key provenance. +- [ ] **Step 3:** Delete origin markers only after locally-originated community copies are removed, except bounded retention reclamation of markers with no exact community bundle copy. +- [ ] **Step 4:** Verify purge still succeeds if the export and publisher key are corrupted/missing. + +### Task 11: Harden diagnostics and privacy scanner + +**Files:** +- Modify: `src/lab/public/strict-json.ts` +- Modify: `src/lab/public/privacy.ts` + +**Interfaces:** +- Produces: constant-size duplicate-key errors and detection of unbracketed IPv6 literals in semantic public strings. + +- [ ] **Step 1:** Stop reflecting attacker-controlled duplicate key names in errors. +- [ ] **Step 2:** Add bounded IPv6-literal recognition without rejecting ordinary colon-bearing public identifiers such as versioned names. + +### Task 12: Move catalog timeout to the flaky test only + +**Files:** +- Modify: `tests/codex-catalog-sync-hardening.test.ts` +- Modify: `scripts/ci/run-bun-test-batches.sh` + +**Interfaces:** +- Produces: one 15-second Bun test timeout on the known degraded-provider case; all neighboring batch tests remain on the default timeout on Linux and macOS uses the same test-local timeout. + +- [ ] **Step 1:** Add `15_000` only to the degraded-provider test definition. +- [ ] **Step 2:** Remove catalog-specific timeout detection/variables from the batch runner. +- [ ] **Step 3:** Run CI-policy regression green. + +### Task 13: Exact-head closure and PR metadata + +**Files:** +- Modify PR #1510 title/body only after runtime verification. + +**Interfaces:** +- Produces: accurate ready-for-review description of CL-10.1 through CL-10.4 with CL-10.5 explicitly blocked. + +- [ ] **Step 1:** Run focused tests, typecheck/privacy/GUI gates via GitHub Actions on the exact final head. +- [ ] **Step 2:** Confirm Cross-platform CI and React Doctor are green on that exact head. +- [ ] **Step 3:** Update PR title to describe the runtime implementation rather than contract-only scope. +- [ ] **Step 4:** Replace the stale body with implemented scope, trust/privacy invariants, validation evidence, and the CL-10.5 hard stop. +- [ ] **Step 5:** Confirm PR remains open, unmerged, and ready for review. diff --git a/docs/superpowers/plans/2026-08-14-cl10-final-review-closure.md b/docs/superpowers/plans/2026-08-14-cl10-final-review-closure.md new file mode 100644 index 0000000000..114f448c12 --- /dev/null +++ b/docs/superpowers/plans/2026-08-14-cl10-final-review-closure.md @@ -0,0 +1,31 @@ +# CL-10 Final Review Closure + +This addendum records the runtime contracts added after the final deep review of PR #1510. It supplements the earlier CL-10 hardening plan and does not expand scope into CL-10.5 remote publishing. + +## Community mutation contention + +Public-evidence mutation remains serialized across processes, but a live non-reclaimable owner is now a fail-fast condition. Synchronous callers receive `PublicEvidenceValidationError` with code `community_cache_busy` instead of blocking the JavaScript agent while polling. + +The management API maps `community_cache_busy` to HTTP `503` and sets `Retry-After: 1`. Other public-evidence validation failures remain client errors. Stale-owner recovery, inode checks, exclusive reclaim claims, and ownership-safe release semantics are unchanged. + +## Sensitive purge semantics + +Once durable local provenance classifies a community cache pathname as locally originated, sensitive purge removes that exact pathname even if the cached object has become oversized, hardlinked, symlinked, or otherwise unreadable through normal community-object validation. + +Deletion uses pathname unlink semantics only. It does not follow a symlink target and does not remove another hardlink to the same inode. `ENOENT` is treated as already absent; other unlink failures remain errors. Origin markers are cleared only after the community deletion pass and directory durability boundary complete. + +## Revocation target errors + +A direct same-publisher bundle revocation whose target bundle is absent is normalized to `PublicEvidenceValidationError` code `revocation_target` with message `revocation target bundle not found`. The optimized direct-target path must not leak platform-specific filesystem `ENOENT` errors. + +## Regression requirements + +The closure is protected by focused tests that require: + +- live lock contention to return `community_cache_busy` in under 500 ms; +- the management community endpoint to return `503` plus `Retry-After: 1` for that contention; +- oversized locally-originated community copies to be removed during sensitive purge; +- hardlinked locally-originated cache pathnames to be removed while a peer hardlink survives; and +- missing direct revocation bundle targets to return stable `revocation_target` errors. + +Exact-head GitHub Actions success is required before this closure is considered verified. PR #1510 must remain open and unmerged during this review cycle. diff --git a/docs/superpowers/specs/2026-08-12-cl10-public-evidence-design.md b/docs/superpowers/specs/2026-08-12-cl10-public-evidence-design.md new file mode 100644 index 0000000000..1495d76ed8 --- /dev/null +++ b/docs/superpowers/specs/2026-08-12-cl10-public-evidence-design.md @@ -0,0 +1,127 @@ +# CL-10 Public Evidence Design + +## Status + +Design approved for contract drafting on 2026-08-12. Independent review accepted the contract on 2026-08-12, and explicit maintainer direction now authorizes CL-10.1 through CL-10.4 runtime implementation on this branch. CL-10.5 remote publishing remains blocked on an exact independently accepted transport/service contract. + +Base: `dev` at `4fed8d3fe431ad23be83f3aff2af18ef8b8ecd71`, the CL-09 merge commit from #1489. + +## Problem + +Compatibility Lab now has local protocol, live-route, task-effectiveness, automatic-refresh, and passive-production evidence. The remaining programme boundary is public sharing. + +Local evidence cannot be published directly because local schemas intentionally contain installation-scoped identity and operational metadata that is safe only inside the local trust domain. Community evidence also cannot be allowed to become canonical local truth merely because a remote bundle is syntactically valid or cryptographically signed. + +## Chosen design + +Use a deterministic, closed public projection with a separate community trust domain. + +```text +local canonical evidence + -> exportability gate + -> allowlist-only public projection + -> export privacy scan + -> export-scoped IDs + -> canonical bundle digest + -> pseudonymous publisher signature + -> explicit local export + -> optional explicit publish after transport contract acceptance + +community bundle + -> bounded parser + -> schema/digest/signature verification + -> non-authoritative community cache + -> clearly labelled read surface + -> never local verdict/routing/scheduling authority +``` + +## Key decisions + +### Public route identity + +A local route is exportable only when its behavior can be represented entirely through entries in the versioned, content-addressed, repo-reviewed `PublicRouteRegistryManifestV1`; dynamic discovery, config, and imported bundles cannot extend that authority. Private/custom endpoint, header, provider-instance, project/location, tenant, account, or custom model/provider dimensions make the route `not_exportable`. + +The exporter must never create a broader public claim by deleting a private dimension from an exact local route subject. + +### Public schema + +`PublicEvidenceBundleV1` is independently versioned and allowlist-only. Dedicated runtime validators enforce its closed types. Records use a layer-matched public subject union rather than assuming every evidence layer is a route. Incident references are closed `IC-NNN` corpus IDs only, and artifact references resolve only to public artifacts in the same bundle. + +Unknown fields fail closed on export and import. + +### IDs + +Local subject, event, artifact, request, decision, and Fabric IDs never leave the installation. Public IDs are derived only from canonical public-safe bytes under explicit domain-separated hashes. + +### Canonical bytes and signatures + +CL-10 V1 freezes RFC 8785 JSON Canonicalization Scheme (JCS) over UTF-8 as the canonical byte representation. Raw imported JSON must be valid UTF-8 and must reject duplicate decoded object member names before semantic object construction, including equivalent escaped spellings such as `"a"` and `"\u0061"`. + +Every public hash is: + +```text +H(domain, value) = SHA-256(UTF8(domain) || 0x00 || UTF8(JCS(value))) +``` + +The exact V1 domains are `ocx-lab-public:subject:v1`, `ocx-lab-public:record:v1`, `ocx-lab-public:bundle:v1`, `ocx-lab-public:bundle-digest:v1`, `ocx-lab-public:artifact:v1`, `ocx-lab-public:publisher-key:v1`, `ocx-lab-public:revocation:v1`, and `ocx-lab-public:route-registry:v1` for their corresponding identities. + +For a bundle, `C = {schemaVersion, exportPolicyVersion, createdDayUtc, publisher, records, artifacts}`. `bundleId = H("ocx-lab-public:bundle:v1", C)`. `bundleDigest = H("ocx-lab-public:bundle-digest:v1", {...C, bundleId})`. Therefore `bundleDigest` and `signature` are excluded from the bundle-digest preimage, and `bundleId`, `bundleDigest`, and `signature` are excluded from the bundle-ID preimage. Ed25519 signs the raw 32 bytes obtained by hex-decoding `bundleDigest`; `signature.signedDigest` must equal `bundleDigest` exactly. + +A revocation similarly hashes `R = {schemaVersion, issuedDayUtc, publisher, targets, reason}` under `ocx-lab-public:revocation:v1`; `revocationId` and `signature` are excluded from `R`, and Ed25519 signs the raw 32 bytes of `revocationId`. Targets are sorted and unique before hashing. + +Import verification order is fixed: byte cap; strict UTF-8 and duplicate-key rejection; JSON syntax/structural bounds; closed schema/version/field validation and publisher-key-ID recomputation; public identity/reference and bundle digest recomputation; `signedDigest` equality; Ed25519 key/signature decoding and verification; repository route/suite/scenario/Fabric authority validation; revocation bootstrap only against an already-verified exact target publisher/bundle; persistence only after every preceding check succeeds. + +### Artifacts + +Artifacts require explicit `public_export` policy. A second export sanitizer and secret/PII scan runs before public artifact hashing. Local visibility alone never authorizes export. + +### Consent + +There is no automatic telemetry. Preview is local and network-free. Export is explicit. Publishing is a second explicit action for a specific bundle and is not implemented until an exact remote-service contract is accepted. + +### Publisher provenance + +Publishable bundles use an installation-local Ed25519 publisher key. The public key provides pseudonymous continuity; the signature proves bundle integrity and signer continuity only. It does not prove that the compatibility claim is true. + +### Community trust + +Imported community evidence is `community_untrusted_v1`. A valid signature produces `cryptographically_valid`, not `locally_verified`. + +Community evidence cannot: + +- append to local `compatibility.jsonl`; +- alter local canonical verdicts or freshness; +- satisfy Routing Profile compatibility requirements; +- influence Router Intelligence; +- trigger CL-08 refresh scheduling; +- merge into a combined local/community score. + +### Revocation + +Publishers can issue signed revocations with finite reason codes. Revocation authority bootstraps from the exact publisher key embedded in the already-verified target bundle; V1 permits no cross-key revocation or key rotation, and duplicate identical revocations are idempotent while conflicting replay fails closed. Consumers suppress revoked records from default community summaries while retaining the audit relation. Remote deletion is transport-specific and does not replace revocation. CL-00 sensitive purge still removes every affected local generated export plus locally-originated community-cache copy fail-closed; local purge never depends on network acknowledgement. + +### Remote service + +Bundle semantics, signing, import, and trust are frozen before any network publishing implementation. A remote publisher requires a reviewed fixed service origin, authentication, TLS/redirect, request-budget, retry/idempotency, retention/deletion, revocation, abuse/rate-limit, and server-validation contract. Arbitrary upload URLs are forbidden. + +## Delivery decomposition + +1. **CL-10.0 Contract:** freeze privacy, exportability, schema, IDs, signatures, consent, trust, revocation, and transport gate. +2. **CL-10.1 Public projector:** closed DTOs, exportability rules, deterministic canonicalization, export privacy validator. +3. **CL-10.2 Bundle/signature substrate:** local exports, publisher-key lifecycle, digest/sign/verify. +4. **CL-10.3 Local surfaces:** preview and explicit local export via existing Lab CLI/API/UI conventions. +5. **CL-10.4 Community import:** bounded import, verification, separate community cache, revocation and labelled read surfaces. +6. **CL-10.5 Remote publishing:** only after exact service contract acceptance. +7. **CL-10.6 Closure:** adversarial privacy/trust tests, cross-platform validation, independent review, programme closure. + +## Validation expectations + +The implementation must include adversarial tests for secret/PII canaries, local IDs, private route dimensions, unknown fields, duplicate JSON object keys, oversized/deep bundles, invalid digest/signature, replay/deduplication, revocation, deterministic export, fixed canonical digest/signature vectors, and complete isolation from local verdicts/routing/CL-08. + +The contract review gate is satisfied. Runtime CL-10.1 through CL-10.4 may now land on this PR under TDD and full validation; CL-10.5 remote publishing remains out of scope. + +## Source of truth + +The detailed normative contract is: + +`devlog/_plan/260807_compatibility_lab/010_cl10_public_evidence_export.md` diff --git a/gui/.eslint/i18n-allowlist.ts b/gui/.eslint/i18n-allowlist.ts index 83974db5a8..acab975933 100644 --- a/gui/.eslint/i18n-allowlist.ts +++ b/gui/.eslint/i18n-allowlist.ts @@ -46,6 +46,7 @@ const TECHNICAL_UNITS = new Set([ "Mo", "Mi", "Fr", + "HTTP", // IEC binary unit rendered next to a formatted number; a unit symbol, not UI prose. "GiB", ]); @@ -97,7 +98,8 @@ export function isTechnicalLiteral(value: string): boolean { if (/^ocx\b/i.test(trimmed)) return true; if (/^codex\b/i.test(trimmed)) return true; - // HTTP headers / auth schemes + // HTTP protocol / headers / auth schemes + if (/^HTTP$/i.test(trimmed)) return true; if (/^Authorization\b/i.test(trimmed)) return true; if (/^Bearer\b/i.test(trimmed)) return true; if (/^Content-Type\b/i.test(trimmed)) return true; diff --git a/gui/src/components/provider-workspace/ProviderSettings.tsx b/gui/src/components/provider-workspace/ProviderSettings.tsx index 26ca85fa35..873d6f0730 100644 --- a/gui/src/components/provider-workspace/ProviderSettings.tsx +++ b/gui/src/components/provider-workspace/ProviderSettings.tsx @@ -14,6 +14,7 @@ import { useT } from "../../i18n/shared"; import { IconLock } from "../../icons"; import { isCatalogProviderId } from "../../provider-icons"; import { openAiAccountProviderState } from "../../provider-payload"; +import { providerSupportsLiveModelDiscovery } from "../../provider-workspace/catalog"; import type { CatalogPreset } from "../provider-catalog/provider-presets"; import { authModeLabel } from "./ProviderRail"; import type { WorkspaceItem, ProviderUpdatePatch } from "./types"; @@ -37,6 +38,8 @@ export default function ProviderSettings({ }) { const t = useT(); const initialAuth = String(item.authMode ?? (item.keyOptional ? "local" : "key")); + const liveModelDiscoverySupported = providerSupportsLiveModelDiscovery(item.name, item); + const savedLiveModels = liveModelDiscoverySupported ? item.liveModels !== false : false; const [adapter, setAdapter] = useState(item.adapter); const [baseUrl, setBaseUrl] = useState(item.baseUrl); const [defaultModel, setDefaultModel] = useState(item.defaultModel ?? ""); @@ -44,7 +47,7 @@ export default function ProviderSettings({ const [apiKeyTransport, setApiKeyTransport] = useState(item.apiKeyTransport ?? "x-api-key"); const [note, setNote] = useState(item.note ?? ""); const [allowPrivateNetwork, setAllowPrivateNetwork] = useState(item.allowPrivateNetwork ?? false); - const [liveModels, setLiveModels] = useState(item.liveModels !== false); + const [liveModels, setLiveModels] = useState(savedLiveModels); const [saving, setSaving] = useState(false); const [msg, setMsg] = useState<{ ok: boolean; text: string } | null>(null); const [accountMode, setAccountMode] = useState<"pool" | "direct">(item.codexAccountMode ?? "pool"); @@ -63,11 +66,11 @@ export default function ProviderSettings({ setApiKeyTransport(item.apiKeyTransport ?? "x-api-key"); setNote(item.note ?? ""); setAllowPrivateNetwork(item.allowPrivateNetwork ?? false); - setLiveModels(item.liveModels !== false); + setLiveModels(savedLiveModels); setMsg(null); setModeMsg(null); queueMicrotask(() => setEndpointChoice(matchChoiceId(baseUrlChoices, item.baseUrl))); - }, [item.adapter, item.baseUrl, item.defaultModel, item.authMode, item.apiKeyTransport, item.keyOptional, item.note, item.allowPrivateNetwork, item.liveModels, baseUrlChoices]); + }, [item.adapter, item.baseUrl, item.defaultModel, item.authMode, item.apiKeyTransport, item.keyOptional, item.note, item.allowPrivateNetwork, savedLiveModels, baseUrlChoices]); /* eslint-enable react-hooks/set-state-in-effect */ // Account mode syncs on its own: a mode PATCH refresh must not reset an in-progress @@ -116,7 +119,7 @@ export default function ProviderSettings({ || (adapter.trim() === "anthropic" && authMode === "key" && apiKeyTransport !== (item.apiKeyTransport ?? "x-api-key")) || note.trim() !== (item.note ?? "") || allowPrivateNetwork !== (item.allowPrivateNetwork ?? false) - || liveModels !== (item.liveModels !== false); + || liveModels !== savedLiveModels; useEffect(() => { onDirtyChange?.(dirty); return () => onDirtyChange?.(false); }, [dirty, onDirtyChange]); @@ -155,7 +158,7 @@ export default function ProviderSettings({ const patch: ProviderUpdatePatch = { adapter: adapter.trim(), baseUrl: nextBaseUrl, defaultModel: defaultModel.trim(), authMode, note: note.trim(), allowPrivateNetwork }; // Keep omitted legacy values omitted unless the user actually changes this toggle. // Otherwise an unrelated settings save manufactures `liveModels: true` provenance. - if (liveModels !== (item.liveModels !== false)) patch.liveModels = liveModels; + if (liveModelDiscoverySupported && liveModels !== (item.liveModels !== false)) patch.liveModels = liveModels; if (supportsApiKeyTransport) patch.apiKeyTransport = apiKeyTransport; else if (item.apiKeyTransport !== undefined) patch.apiKeyTransport = ""; const res = await onUpdateProvider(item.name, patch); @@ -200,7 +203,7 @@ export default function ProviderSettings({ setAdapter(item.adapter); setBaseUrl(item.baseUrl); setDefaultModel(item.defaultModel ?? ""); setAuthMode(initialAuth); setApiKeyTransport(item.apiKeyTransport ?? "x-api-key"); - setNote(item.note ?? ""); setAllowPrivateNetwork(item.allowPrivateNetwork ?? false); setLiveModels(item.liveModels !== false); setMsg(null); + setNote(item.note ?? ""); setAllowPrivateNetwork(item.allowPrivateNetwork ?? false); setLiveModels(savedLiveModels); setMsg(null); setEndpointChoice(matchChoiceId(baseUrlChoices, item.baseUrl)); }; @@ -335,7 +338,12 @@ export default function ProviderSettings({ <span className="pwi-settings-label">{t("pws.allowPrivateNetwork")}</span> </label> <label className="pwi-settings-field" style={{ flexDirection: "row", alignItems: "flex-start", gap: 8 }}> - <input type="checkbox" checked={liveModels} onChange={e => setLiveModels(e.target.checked)} /> + <input + type="checkbox" + checked={liveModels} + disabled={!liveModelDiscoverySupported} + onChange={e => setLiveModels(e.target.checked)} + /> <span> <span className="pwi-settings-label">{t("pws.liveModels")}</span> <span className="muted text-label" style={{ display: "block", marginTop: 2 }}>{t("pws.liveModelsDesc")}</span> diff --git a/gui/src/i18n/de.ts b/gui/src/i18n/de.ts index 5b8d39ac6b..b3c60d9232 100644 --- a/gui/src/i18n/de.ts +++ b/gui/src/i18n/de.ts @@ -554,6 +554,15 @@ export const de: Record<TKey, string> = { "models.customFieldDisplayNamePlaceholder": "z. B. Qwen 4 Max Preview", "models.customFieldContext": "Kontextfenster", "models.customFieldModalities": "Eingabemodalitäten", + "models.customFieldReasoning": "Reasoning-Aufwand", + "models.customFieldReasoningOverride": "Reasoning-Aufwand überschreiben", + "models.reasoningEffort.none": "Keine", + "models.reasoningEffort.minimal": "Minimal", + "models.reasoningEffort.low": "Niedrig", + "models.reasoningEffort.medium": "Mittel", + "models.reasoningEffort.high": "Hoch", + "models.reasoningEffort.xhigh": "Sehr hoch", + "models.reasoningEffort.max": "Maximal", "models.tipProvider": "Anbieter", "models.tipContext": "Kontext", "models.tipModalities": "Modalitäten", diff --git a/gui/src/i18n/en.ts b/gui/src/i18n/en.ts index 867c27c079..04862b599d 100644 --- a/gui/src/i18n/en.ts +++ b/gui/src/i18n/en.ts @@ -579,6 +579,15 @@ export const en = { "models.customFieldDisplayNamePlaceholder": "e.g. Qwen 4 Max Preview", "models.customFieldContext": "Context window", "models.customFieldModalities": "Input modalities", + "models.customFieldReasoning": "Reasoning effort", + "models.customFieldReasoningOverride": "Override reasoning effort", + "models.reasoningEffort.none": "None", + "models.reasoningEffort.minimal": "Minimal", + "models.reasoningEffort.low": "Low", + "models.reasoningEffort.medium": "Medium", + "models.reasoningEffort.high": "High", + "models.reasoningEffort.xhigh": "Extra high", + "models.reasoningEffort.max": "Maximum", "models.tipProvider": "Provider", "models.tipContext": "Context", "models.tipModalities": "Modalities", diff --git a/gui/src/i18n/ja.ts b/gui/src/i18n/ja.ts index 5297e0119c..49b3187ba6 100644 --- a/gui/src/i18n/ja.ts +++ b/gui/src/i18n/ja.ts @@ -1972,6 +1972,15 @@ export const ja: Record<TKey, string> = { "models.customFieldDisplayNamePlaceholder": "e.g. Qwen 4 Max Preview", "models.customFieldContext": "Context window", "models.customFieldModalities": "Input modalities", + "models.customFieldReasoning": "推論努力", + "models.customFieldReasoningOverride": "推論努力を上書き", + "models.reasoningEffort.none": "なし", + "models.reasoningEffort.minimal": "最小", + "models.reasoningEffort.low": "低", + "models.reasoningEffort.medium": "中", + "models.reasoningEffort.high": "高", + "models.reasoningEffort.xhigh": "非常に高", + "models.reasoningEffort.max": "最大", "models.tipProvider": "Provider", "models.tipContext": "Context", "models.tipModalities": "Modalities", diff --git a/gui/src/i18n/ko.ts b/gui/src/i18n/ko.ts index c4103b9dda..03aa89a664 100644 --- a/gui/src/i18n/ko.ts +++ b/gui/src/i18n/ko.ts @@ -565,6 +565,15 @@ export const ko: Record<TKey, string> = { "models.customFieldDisplayNamePlaceholder": "예: Qwen 4 Max Preview", "models.customFieldContext": "컨텍스트 윈도우", "models.customFieldModalities": "입력 모달리티", + "models.customFieldReasoning": "추론 노력", + "models.customFieldReasoningOverride": "추론 노력 재정의", + "models.reasoningEffort.none": "없음", + "models.reasoningEffort.minimal": "최소", + "models.reasoningEffort.low": "낮음", + "models.reasoningEffort.medium": "중간", + "models.reasoningEffort.high": "높음", + "models.reasoningEffort.xhigh": "매우 높음", + "models.reasoningEffort.max": "최대", "models.tipProvider": "프로바이더", "models.tipContext": "컨텍스트", "models.tipModalities": "모달리티", diff --git a/gui/src/i18n/lab-translations.ts b/gui/src/i18n/lab-translations.ts index 3428a847dd..75a21feb4e 100644 --- a/gui/src/i18n/lab-translations.ts +++ b/gui/src/i18n/lab-translations.ts @@ -7,7 +7,12 @@ export type LabSupplementKey = | "artifact.present" | "artifact.corrupt" | "artifact.purged_unavailable" - | "selectVerdict"; + | "selectVerdict" + | "community.title" + | "community.notLocalVerdict" + | "community.bundles" + | "community.activeRecords" + | "community.revokedRecords"; const en: Record<LabCatalogKey, string> = { "lab.title": "Compatibility Lab", @@ -427,6 +432,11 @@ const supplements: Record<LabLocale, Record<LabSupplementKey, string>> = { "artifact.corrupt": "Corrupt", "artifact.purged_unavailable": "Purged / unavailable", selectVerdict: "View verdict for {subject}", + "community.title": "Community evidence", + "community.notLocalVerdict": "Untrusted read-only context. Not included in this local verdict.", + "community.bundles": "Bundles", + "community.activeRecords": "Active records", + "community.revokedRecords": "Revoked records", }, de: { subjectKindUnknown: "Unbekannt", @@ -434,6 +444,11 @@ const supplements: Record<LabLocale, Record<LabSupplementKey, string>> = { "artifact.corrupt": "Beschädigt", "artifact.purged_unavailable": "Gelöscht / nicht verfügbar", selectVerdict: "Urteil für {subject} anzeigen", + "community.title": "Community-Evidenz", + "community.notLocalVerdict": "Nicht vertrauenswürdiger Nur-Lese-Kontext. Nicht Teil dieses lokalen Urteils.", + "community.bundles": "Pakete", + "community.activeRecords": "Aktive Einträge", + "community.revokedRecords": "Widerrufene Einträge", }, ko: { subjectKindUnknown: "알 수 없음", @@ -441,6 +456,11 @@ const supplements: Record<LabLocale, Record<LabSupplementKey, string>> = { "artifact.corrupt": "손상됨", "artifact.purged_unavailable": "삭제됨 / 사용할 수 없음", selectVerdict: "{subject}의 판정 보기", + "community.title": "커뮤니티 증거", + "community.notLocalVerdict": "신뢰되지 않는 읽기 전용 컨텍스트입니다. 이 로컬 판정에는 포함되지 않습니다.", + "community.bundles": "번들", + "community.activeRecords": "활성 레코드", + "community.revokedRecords": "폐기된 레코드", }, zh: { subjectKindUnknown: "未知", @@ -448,6 +468,11 @@ const supplements: Record<LabLocale, Record<LabSupplementKey, string>> = { "artifact.corrupt": "已损坏", "artifact.purged_unavailable": "已清除 / 不可用", selectVerdict: "查看 {subject} 的判定", + "community.title": "社区证据", + "community.notLocalVerdict": "不受信任的只读上下文。不计入此本地判定。", + "community.bundles": "证据包", + "community.activeRecords": "有效记录", + "community.revokedRecords": "已撤销记录", }, "zh-TW": { subjectKindUnknown: "未知", @@ -455,6 +480,11 @@ const supplements: Record<LabLocale, Record<LabSupplementKey, string>> = { "artifact.corrupt": "已損壞", "artifact.purged_unavailable": "已清除 / 不可用", selectVerdict: "查看 {subject} 的判定", + "community.title": "社群證據", + "community.notLocalVerdict": "不受信任的唯讀脈絡。不計入此本地判定。", + "community.bundles": "證據包", + "community.activeRecords": "有效記錄", + "community.revokedRecords": "已撤銷記錄", }, ru: { subjectKindUnknown: "Неизвестно", @@ -462,6 +492,11 @@ const supplements: Record<LabLocale, Record<LabSupplementKey, string>> = { "artifact.corrupt": "Повреждён", "artifact.purged_unavailable": "Удалён / недоступен", selectVerdict: "Открыть вердикт для {subject}", + "community.title": "Данные сообщества", + "community.notLocalVerdict": "Недоверенный контекст только для чтения. Не входит в этот локальный вердикт.", + "community.bundles": "Пакеты", + "community.activeRecords": "Активные записи", + "community.revokedRecords": "Отозванные записи", }, ja: { subjectKindUnknown: "不明", @@ -469,6 +504,11 @@ const supplements: Record<LabLocale, Record<LabSupplementKey, string>> = { "artifact.corrupt": "破損", "artifact.purged_unavailable": "削除済み / 利用不可", selectVerdict: "{subject} の判定を表示", + "community.title": "コミュニティ証拠", + "community.notLocalVerdict": "信頼されていない読み取り専用コンテキストです。このローカル判定には含まれません。", + "community.bundles": "バンドル", + "community.activeRecords": "有効なレコード", + "community.revokedRecords": "取り消されたレコード", }, tr: { subjectKindUnknown: "Bilinmiyor", @@ -476,6 +516,11 @@ const supplements: Record<LabLocale, Record<LabSupplementKey, string>> = { "artifact.corrupt": "Bozuk", "artifact.purged_unavailable": "Temizlenmiş / kullanılamıyor", selectVerdict: "{subject} için kararı görüntüle", + "community.title": "Topluluk kanıtı", + "community.notLocalVerdict": "Güvenilmeyen salt okunur bağlam. Bu yerel karara dahil değildir.", + "community.bundles": "Paketler", + "community.activeRecords": "Etkin kayıtlar", + "community.revokedRecords": "Geri çekilen kayıtlar", }, }; diff --git a/gui/src/i18n/ru.ts b/gui/src/i18n/ru.ts index 42a606fea7..965e50703a 100644 --- a/gui/src/i18n/ru.ts +++ b/gui/src/i18n/ru.ts @@ -567,6 +567,15 @@ export const ru: Record<TKey, string> = { "models.customFieldDisplayNamePlaceholder": "например, Qwen 4 Max Preview", "models.customFieldContext": "Контекстное окно", "models.customFieldModalities": "Входные модальности", + "models.customFieldReasoning": "Уровень рассуждений", + "models.customFieldReasoningOverride": "Переопределить уровень рассуждений", + "models.reasoningEffort.none": "Нет", + "models.reasoningEffort.minimal": "Минимальный", + "models.reasoningEffort.low": "Низкий", + "models.reasoningEffort.medium": "Средний", + "models.reasoningEffort.high": "Высокий", + "models.reasoningEffort.xhigh": "Очень высокий", + "models.reasoningEffort.max": "Максимальный", "models.tipProvider": "Провайдер", "models.tipContext": "Контекст", "models.tipModalities": "Модальности", diff --git a/gui/src/i18n/tr.ts b/gui/src/i18n/tr.ts index b352fb6917..0cb388d60e 100644 --- a/gui/src/i18n/tr.ts +++ b/gui/src/i18n/tr.ts @@ -570,6 +570,15 @@ export const tr: Record<TKey, string> = { "models.customFieldDisplayNamePlaceholder": "örn. Qwen 4 Max Preview", "models.customFieldContext": "Bağlam penceresi", "models.customFieldModalities": "Girdi türleri", + "models.customFieldReasoning": "Akıl yürütme çabası", + "models.customFieldReasoningOverride": "Akıl yürütme çabasını geçersiz kıl", + "models.reasoningEffort.none": "Yok", + "models.reasoningEffort.minimal": "Minimal", + "models.reasoningEffort.low": "Düşük", + "models.reasoningEffort.medium": "Orta", + "models.reasoningEffort.high": "Yüksek", + "models.reasoningEffort.xhigh": "Çok yüksek", + "models.reasoningEffort.max": "Maksimum", "models.tipProvider": "Sağlayıcı", "models.tipContext": "Bağlam", "models.tipModalities": "Girdi Türleri", diff --git a/gui/src/i18n/zh-TW.ts b/gui/src/i18n/zh-TW.ts index 96c34592e9..1f24a0f8de 100644 --- a/gui/src/i18n/zh-TW.ts +++ b/gui/src/i18n/zh-TW.ts @@ -435,6 +435,15 @@ export const zhTW: Record<TKey, string> = { "models.customFieldDisplayNamePlaceholder": "例如 Qwen 4 Max Preview", "models.customFieldContext": "上下文視窗", "models.customFieldModalities": "輸入模態", + "models.customFieldReasoning": "推理強度", + "models.customFieldReasoningOverride": "覆寫推理強度", + "models.reasoningEffort.none": "無", + "models.reasoningEffort.minimal": "最低", + "models.reasoningEffort.low": "低", + "models.reasoningEffort.medium": "中", + "models.reasoningEffort.high": "高", + "models.reasoningEffort.xhigh": "極高", + "models.reasoningEffort.max": "最高", "models.tipProvider": "供應商", "models.tipContext": "上下文", "models.tipModalities": "模態", diff --git a/gui/src/i18n/zh.ts b/gui/src/i18n/zh.ts index f0c42fbc26..88f6a5554b 100644 --- a/gui/src/i18n/zh.ts +++ b/gui/src/i18n/zh.ts @@ -562,6 +562,15 @@ export const zh: Record<TKey, string> = { "models.customFieldDisplayNamePlaceholder": "例如 Qwen 4 Max Preview", "models.customFieldContext": "上下文窗口", "models.customFieldModalities": "输入模态", + "models.customFieldReasoning": "推理强度", + "models.customFieldReasoningOverride": "覆盖推理强度", + "models.reasoningEffort.none": "无", + "models.reasoningEffort.minimal": "最低", + "models.reasoningEffort.low": "低", + "models.reasoningEffort.medium": "中", + "models.reasoningEffort.high": "高", + "models.reasoningEffort.xhigh": "极高", + "models.reasoningEffort.max": "最高", "models.tipProvider": "提供方", "models.tipContext": "上下文", "models.tipModalities": "模态", diff --git a/gui/src/pages/CompatibilityMatrix.tsx b/gui/src/pages/CompatibilityMatrix.tsx index 9fcac9b129..c8958cabbc 100644 --- a/gui/src/pages/CompatibilityMatrix.tsx +++ b/gui/src/pages/CompatibilityMatrix.tsx @@ -9,6 +9,7 @@ import { fetchLabPageData, fetchMoreVerdicts, fetchVerdictDetail, + type CommunityEvidenceContextDto, type LabPageData, type VerdictDetailData, } from "./compatibility-matrix-api"; @@ -62,6 +63,12 @@ type ExtraVerdictPage = { hasMore: boolean; }; +type LoadMoreFailure = { + baseData: LabPageData; + queryKey: string; + message: string; +}; + function localizedFetchError(e: unknown, fallback: string): string { if (!(e instanceof Error)) return fallback; const msg = e.message; @@ -71,13 +78,7 @@ function localizedFetchError(e: unknown, fallback: string): string { return msg || fallback; } -function VerdictBadge({ - verdict, - caption, - label, - selected, - onSelect, -}: { +function VerdictBadge({ verdict, caption, label, selected, onSelect }: { verdict: CompatibilityVerdict; caption: string; label: string; @@ -101,12 +102,7 @@ function VerdictBadge({ ); } -function VerdictCell({ - rows, - t, - selectedKey, - onSelect, -}: { +function VerdictCell({ rows, t, selectedKey, onSelect }: { rows: VerdictDto[]; t: (key: TKey) => string; selectedKey: string | null; @@ -154,15 +150,27 @@ function StatusCards({ data, t, locale }: { ); } -function DetailPane({ - verdict, - detail, - loading, - error, - t, - locale, - onClose, -}: { +function CommunityEvidencePanel({ community, locale }: { + community: CommunityEvidenceContextDto | null; + locale: Parameters<typeof labSupplement>[0]; +}) { + if (!community || community.evidence.length === 0) return null; + const activeRecords = community.evidence.reduce((total, row) => total + row.activeRecordCount, 0); + const revokedRecords = community.evidence.reduce((total, row) => total + row.revokedRecordCount, 0); + return ( + <section className="lab-matrix-block" data-testid="lab-community-evidence"> + <h3 className="lab-matrix-title">{labSupplement(locale, "community.title")}</h3> + <p className="muted">{labSupplement(locale, "community.notLocalVerdict")}</p> + <dl className="lab-detail-meta"> + <div><dt>{labSupplement(locale, "community.bundles")}</dt><dd>{community.evidence.length}</dd></div> + <div><dt>{labSupplement(locale, "community.activeRecords")}</dt><dd>{activeRecords}</dd></div> + <div><dt>{labSupplement(locale, "community.revokedRecords")}</dt><dd>{revokedRecords}</dd></div> + </dl> + </section> + ); +} + +function DetailPane({ verdict, detail, loading, error, t, locale, onClose }: { verdict: VerdictDto; detail: VerdictDetailData | null; loading: boolean; @@ -171,6 +179,11 @@ function DetailPane({ locale: Parameters<typeof labSupplement>[0]; onClose: () => void; }) { + const expectedEventCount = new Set([ + ...verdict.contributingEventIds, + ...verdict.contradictingEventIds, + ]).size; + return ( <aside className="lab-detail-pane" aria-label={t("lab.detailTitle")}> <div className="lab-detail-head"> @@ -219,9 +232,12 @@ function DetailPane({ </ul> </section> )} - {detail.events.length > 0 && ( + {(detail.events.length > 0 || expectedEventCount > 0) && ( <section className="lab-detail-section"> - <h4>{t("lab.detailEvents")}</h4> + <h4> + {t("lab.detailEvents")} + {detail.events.length < expectedEventCount ? ` (${detail.events.length}/${expectedEventCount})` : ""} + </h4> <ul className="lab-detail-list"> {detail.events.map(event => ( <li key={event.eventId}> @@ -252,11 +268,7 @@ function DetailPane({ ); } -export default function CompatibilityMatrix({ - apiBase, - active = true, - onCountChange, -}: { +export default function CompatibilityMatrix({ apiBase, active = true, onCountChange }: { apiBase: string; active?: boolean; onCountChange?: (count: number | null) => void; @@ -264,6 +276,7 @@ export default function CompatibilityMatrix({ const { t, locale } = useI18n(); const [filters, setFilters] = useState<VerdictFilters>({ layer: "", verdict: "", subjectQuery: "", suiteId: "" }); const [extraPage, setExtraPage] = useState<ExtraVerdictPage | null>(null); + const [loadMoreFailure, setLoadMoreFailure] = useState<LoadMoreFailure | null>(null); const [loadingMore, setLoadingMore] = useState(false); const [selectedVerdict, setSelectedVerdict] = useState<VerdictDto | null>(null); const [detail, setDetail] = useState<VerdictDetailData | null>(null); @@ -276,7 +289,6 @@ export default function CompatibilityMatrix({ const queryFilters = useMemo(() => verdictQueryFromFilters(filters), [filters]); const queryKey = JSON.stringify(queryFilters); - const fetchPage = useCallback( (signal: AbortSignal) => fetchLabPageData(apiBase, queryFilters, signal), [apiBase, queryFilters], @@ -293,6 +305,7 @@ export default function CompatibilityMatrix({ loadMoreRef.current?.abort(); loadMoreRef.current = null; setExtraPage(null); + setLoadMoreFailure(null); setLoadingMore(false); }, []); @@ -306,13 +319,7 @@ export default function CompatibilityMatrix({ setDetailLoading(false); }, []); - useEffect(() => { - // A refreshed first page makes any in-flight cursor request stale. The associated - // appended-page state is identity-bound below, so it becomes invisible immediately - // without synchronously cascading state from this effect. - loadMoreRef.current?.abort(); - }, [surface.data]); - + useEffect(() => { loadMoreRef.current?.abort(); }, [surface.data]); useEffect(() => () => { loadMoreRef.current?.abort(); detailRequestRef.current?.abort(); @@ -329,13 +336,16 @@ export default function CompatibilityMatrix({ && extraPage.queryKey === queryKey ? extraPage : null; + const visibleLoadMoreError = loadMoreFailure !== null + && loadMoreFailure.baseData === surface.data + && loadMoreFailure.queryKey === queryKey + ? loadMoreFailure.message + : null; const reportedCount = useMemo(() => { if (!active || !surface.data?.status.projectionAvailable) return null; const total = surface.data.status.verdictCount; - return typeof total === "number" - ? total - : surface.data.verdicts.length + (validExtraPage?.verdicts.length ?? 0); + return typeof total === "number" ? total : surface.data.verdicts.length + (validExtraPage?.verdicts.length ?? 0); }, [active, surface.data, validExtraPage]); useEffect(() => { onCountChange?.(reportedCount); }, [onCountChange, reportedCount]); @@ -357,14 +367,13 @@ export default function CompatibilityMatrix({ const controller = new AbortController(); loadMoreRef.current = controller; const startedKey = queryKey; + setLoadMoreFailure(null); setLoadingMore(true); try { const page = await fetchMoreVerdicts(apiBase, queryFilters, cursor, controller.signal); if (controller.signal.aborted) return; setExtraPage(current => { - const existing = current?.baseData === baseData && current.queryKey === startedKey - ? current.verdicts - : []; + const existing = current?.baseData === baseData && current.queryKey === startedKey ? current.verdicts : []; return { baseData, queryKey: startedKey, @@ -373,15 +382,21 @@ export default function CompatibilityMatrix({ hasMore: page.hasMore, }; }); - } catch { - // Keep the current rows. The normal refresh action retries from a consistent first page. + } catch (e) { + if (!controller.signal.aborted) { + setLoadMoreFailure({ + baseData, + queryKey: startedKey, + message: localizedFetchError(e, t("lab.loadFailed")), + }); + } } finally { if (loadMoreRef.current === controller) { loadMoreRef.current = null; setLoadingMore(false); } } - }, [apiBase, loadingMore, queryFilters, queryKey, surface.data, validExtraPage]); + }, [apiBase, loadingMore, queryFilters, queryKey, surface.data, t, validExtraPage]); const selectVerdict = useCallback(async (verdict: VerdictDto) => { if (!active) return; @@ -455,6 +470,7 @@ export default function CompatibilityMatrix({ {loadError && <Notice tone="err">{loadError}</Notice>} {projectionIncompatible && <Notice tone="err">{t("lab.projectionIncompatible")}</Notice>} {projectionUnavailable && !projectionIncompatible && <EmptyState title={t("lab.projectionUnavailable")} />} + {surface.data && <CommunityEvidencePanel community={surface.data.community} locale={locale} />} {surface.data && status?.projectionAvailable && !projectionIncompatible && ( <div className="lab-layout"> @@ -582,6 +598,7 @@ export default function CompatibilityMatrix({ </div> </div> + {visibleLoadMoreError && <Notice tone="err">{visibleLoadMoreError}</Notice>} {pageHasMore && ( <div className="lab-load-more"> <button type="button" className="btn btn-ghost" disabled={loadingMore} onClick={() => { void loadMore(); }}> diff --git a/gui/src/pages/Models.tsx b/gui/src/pages/Models.tsx index 60deb7801a..f2fab8604a 100644 --- a/gui/src/pages/Models.tsx +++ b/gui/src/pages/Models.tsx @@ -53,6 +53,7 @@ import { THREAD_OPTIONS, writeCollapsedProviders, discoveryFailureLabel, + REASONING_EFFORT_LEVELS, type ModelRow, type ProviderContextCapsResponse, type ShadowCallData, @@ -234,6 +235,13 @@ export default function Models({ apiBase, restartEpoch = 0 }: { apiBase: string; const [customFormContextWindow, setCustomFormContextWindow] = useState(""); const [customFormShowCustomCtx, setCustomFormShowCustomCtx] = useState(false); const [customFormModalities, setCustomFormModalities] = useState<string[]>(["text"]); + const [customFormReasoning, setCustomFormReasoning] = useState(false); + const [customFormReasoningEfforts, setCustomFormReasoningEfforts] = useState<string[]>([]); + // Whether the ladder has been seeded at least once. `[]` is a MEANINGFUL explicit + // no-reasoning override, so initialization is tracked separately from the array contents: + // once seeded (an edit's stored ladder — including an explicit empty one — or a new form's + // first enable), re-enabling the override preserves the current array even when empty. + const customFormReasoningInitializedRef = useRef(false); const [customSaving, setCustomSaving] = useState(false); const [customError, setCustomError] = useState(""); const [contextModalProvider, setContextModalProvider] = useState<string | null>(null); @@ -872,6 +880,7 @@ export default function Models({ apiBase, restartEpoch = 0 }: { apiBase: string; displayName?: string, contextWindow?: number, inputModalities?: string[], + reasoningEfforts?: string[], ) => { setCustomSaving(true); setCustomError(""); @@ -879,7 +888,7 @@ export default function Models({ apiBase, restartEpoch = 0 }: { apiBase: string; const r = await fetch(`${apiBase}/api/custom-models`, { method: "POST", headers: { "Content-Type": "application/json" }, - body: JSON.stringify({ provider, modelId, displayName, contextWindow, inputModalities }), + body: JSON.stringify({ provider, modelId, displayName, contextWindow, inputModalities, reasoningEfforts }), }); try { await readJsonOrThrow(r, t("models.customSaveFailed")); @@ -1041,6 +1050,9 @@ export default function Models({ apiBase, restartEpoch = 0 }: { apiBase: string; setCustomFormContextWindow(""); setCustomFormShowCustomCtx(false); setCustomFormModalities(["text"]); + setCustomFormReasoning(false); + setCustomFormReasoningEfforts([]); + customFormReasoningInitializedRef.current = false; setCustomError(""); setCustomModalOpen(true); }} @@ -1187,6 +1199,14 @@ export default function Models({ apiBase, restartEpoch = 0 }: { apiBase: string; setCustomFormContextWindow(m.contextWindow ? String(m.contextWindow) : ""); setCustomFormShowCustomCtx(false); setCustomFormModalities(m.inputModalities ?? ["text"]); + // Only a STORED ladder counts as "configured": an inherited one + // would show a phantom override that saves "inherit" over the + // provider row's current metadata. + setCustomFormReasoning(Array.isArray(m.reasoningEfforts)); + setCustomFormReasoningEfforts(m.reasoningEfforts ?? []); + // A stored ladder — even an explicit empty one — is a real + // configuration: re-enabling must preserve it, not reseed. + customFormReasoningInitializedRef.current = Array.isArray(m.reasoningEfforts); setCustomError(""); setCustomModalOpen(true); setHoveredModel(null); @@ -1636,6 +1656,56 @@ export default function Models({ apiBase, restartEpoch = 0 }: { apiBase: string; ))} </div> </div> + + <div className="text-label models-field"> + {t("models.customFieldReasoning")} + <div className="row models-field-row"> + <label className="row models-modality-option"> + <input + type="checkbox" + checked={customFormReasoning} + onChange={e => { + setCustomFormReasoning(e.target.checked); + if (e.target.checked && !customFormReasoningInitializedRef.current) { + customFormReasoningInitializedRef.current = true; + // First enable: seed from the model's advertised ladder when the + // row is known (a provider may support only a subset of levels — + // preselecting the full shared list would persist levels the model + // does not accept). Unknown model ids fall back to the full set: + // the common intent of enabling the override is "allow every known + // step", and the wire clamp still bounds what is actually sent. + const row = models.find(m => m.provider === customModalProvider && m.id === customFormModelId); + const advertised = Array.isArray(row?.reasoningEfforts) + ? row.reasoningEfforts + : undefined; + setCustomFormReasoningEfforts(advertised ?? [...REASONING_EFFORT_LEVELS]); + } + }} + disabled={customSaving} + /> + <span className="text-control">{t("models.customFieldReasoningOverride")}</span> + </label> + </div> + {customFormReasoning && ( + <div className="row models-field-row" style={{ flexWrap: "wrap" }}> + {REASONING_EFFORT_LEVELS.map(effort => ( + <label key={effort} className="row models-modality-option"> + <input + type="checkbox" + checked={customFormReasoningEfforts.includes(effort)} + onChange={e => { + setCustomFormReasoningEfforts(prev => ( + e.target.checked ? [...prev, effort] : prev.filter(level => level !== effort) + )); + }} + disabled={customSaving} + /> + <span className="text-control">{t(`models.reasoningEffort.${effort}` as TKey)}</span> + </label> + ))} + </div> + )} + </div> </div> <div className="modal-actions"> @@ -1652,19 +1722,24 @@ export default function Models({ apiBase, restartEpoch = 0 }: { apiBase: string; const ctxVal = customFormContextWindow ? Number(customFormContextWindow.replace(/[_,\s]/g, "")) : undefined; const contextWindow = ctxVal && ctxVal > 0 ? Math.floor(ctxVal) : undefined; if (customModalMode === "add") { + const reasoningEfforts = customFormReasoning ? customFormReasoningEfforts : undefined; void addCustomModel( customModalProvider, modelId, displayName || undefined, contextWindow, customFormModalities.length > 0 ? customFormModalities : undefined, + reasoningEfforts, ); } else { + // `null` clears a stored override back to "inherit from the provider row"; + // an explicit empty ladder stays stored as "no reasoning". void updateCustomModel(customModalId, { modelId, displayName, contextWindow: contextWindow ?? null, inputModalities: customFormModalities, + reasoningEfforts: customFormReasoning ? customFormReasoningEfforts : null, }); } }} diff --git a/gui/src/pages/compatibility-matrix-api.ts b/gui/src/pages/compatibility-matrix-api.ts index 139ba8594a..7e28030977 100644 --- a/gui/src/pages/compatibility-matrix-api.ts +++ b/gui/src/pages/compatibility-matrix-api.ts @@ -132,19 +132,14 @@ async function collectPages<T>( seen.add(next); cursor = next; } - // The server kept advancing correctly but exceeded the browser-side safety bound. - // Preserve the coherent prefix and report truncation separately instead of - // misclassifying a legitimate large dataset as a broken pagination contract. return { rows, truncated: true }; } export async function fetchAllSubjects(apiBase: string, signal: AbortSignal): Promise<CollectedPages<SubjectListItemDto>> { - return collectPages( - async cursor => { - const page = await fetchSubjectPage(apiBase, cursor, signal); - return { rows: page.subjects, hasMore: page.hasMore, nextCursor: page.nextCursor }; - }, - ); + return collectPages(async cursor => { + const page = await fetchSubjectPage(apiBase, cursor, signal); + return { rows: page.subjects, hasMore: page.hasMore, nextCursor: page.nextCursor }; + }); } export async function fetchSubjectDetail( @@ -181,12 +176,10 @@ async function fetchAllObservations( filters: { subjectId: string; layer?: string; suiteId?: string }, signal: AbortSignal, ): Promise<CollectedPages<ObservationDto>> { - return collectPages( - async cursor => { - const page = await fetchObservationsPage(apiBase, filters, cursor, signal); - return { rows: page.observations, hasMore: page.hasMore, nextCursor: page.nextCursor }; - }, - ); + return collectPages(async cursor => { + const page = await fetchObservationsPage(apiBase, filters, cursor, signal); + return { rows: page.observations, hasMore: page.hasMore, nextCursor: page.nextCursor }; + }); } export async function fetchEventById(apiBase: string, eventId: string, signal: AbortSignal): Promise<LabEventDto> { @@ -248,6 +241,80 @@ export async function fetchPassiveProductionSummary( return parsePassiveProductionSummary(raw); } +export type CommunityEvidenceSummaryRowDto = { + trustClass: "community_untrusted_v1"; + status: "cryptographically_valid"; + bundleId: string; + publisherKeyId: string; + activeRecordCount: number; + revokedRecordCount: number; +}; + +export type CommunityEvidenceContextDto = { + evidence: CommunityEvidenceSummaryRowDto[]; + trustClass: "community_untrusted_v1"; + locallyVerified: false; +}; + +function hasOnlyKeys(raw: Record<string, unknown>, allowed: readonly string[]): boolean { + const allowedSet = new Set(allowed); + return Object.keys(raw).every(key => allowedSet.has(key)); +} + +function isSha256Hex(value: unknown): value is string { + return typeof value === "string" && /^[0-9a-f]{64}$/.test(value); +} + +function isNonNegativeInteger(value: unknown): value is number { + return typeof value === "number" && Number.isSafeInteger(value) && value >= 0; +} + +export function parseCommunityEvidenceContext(raw: unknown): CommunityEvidenceContextDto | null { + if (!isPlainObject(raw) + || !hasOnlyKeys(raw, ["evidence", "trustClass", "locallyVerified"]) + || raw.trustClass !== "community_untrusted_v1" + || raw.locallyVerified !== false + || !Array.isArray(raw.evidence) + || raw.evidence.length > 4096) { + return null; + } + const evidence: CommunityEvidenceSummaryRowDto[] = []; + for (const value of raw.evidence) { + if (!isPlainObject(value) + || !hasOnlyKeys(value, [ + "trustClass", "status", "bundleId", "publisherKeyId", + "activeRecordCount", "revokedRecordCount", + ]) + || value.trustClass !== "community_untrusted_v1" + || value.status !== "cryptographically_valid" + || !isSha256Hex(value.bundleId) + || !isSha256Hex(value.publisherKeyId) + || !isNonNegativeInteger(value.activeRecordCount) + || !isNonNegativeInteger(value.revokedRecordCount)) { + return null; + } + evidence.push({ + trustClass: "community_untrusted_v1", + status: "cryptographically_valid", + bundleId: value.bundleId, + publisherKeyId: value.publisherKeyId, + activeRecordCount: value.activeRecordCount, + revokedRecordCount: value.revokedRecordCount, + }); + } + return { evidence, trustClass: "community_untrusted_v1", locallyVerified: false }; +} + +export async function fetchCommunityEvidenceContext( + apiBase: string, + signal: AbortSignal, +): Promise<CommunityEvidenceContextDto> { + const raw = await fetchLabJson<unknown>(apiBase, "/api/lab/public/community", signal); + const context = parseCommunityEvidenceContext(raw); + if (!context) throw invalidResponse(); + return context; +} + export type LabPageData = { status: LabStatusDto; verdicts: VerdictDto[]; @@ -255,6 +322,7 @@ export type LabPageData = { subjectsTruncated: boolean; hasMore: boolean; nextCursor?: string; + community: CommunityEvidenceContextDto | null; }; export async function fetchLabPageData( @@ -262,9 +330,15 @@ export async function fetchLabPageData( filters: VerdictQueryFilters, signal: AbortSignal, ): Promise<LabPageData> { - const status = await fetchLabStatus(apiBase, signal); + const [status, community] = await Promise.all([ + fetchLabStatus(apiBase, signal), + fetchCommunityEvidenceContext(apiBase, signal).catch(error => { + if (signal.aborted) throw error; + return null; + }), + ]); if (!status.projectionAvailable) { - return { status, verdicts: [], subjects: [], subjectsTruncated: false, hasMore: false }; + return { status, verdicts: [], subjects: [], subjectsTruncated: false, hasMore: false, community }; } const [verdictPage, subjects] = await Promise.all([ fetchVerdictPage(apiBase, filters, undefined, signal), @@ -277,6 +351,7 @@ export async function fetchLabPageData( subjectsTruncated: subjects.truncated, hasMore: verdictPage.hasMore, nextCursor: verdictPage.nextCursor, + community, }; } @@ -316,7 +391,6 @@ async function mapSettledBounded<TItem, TResult>( results.push(await mapper(limited[current]!)); } catch (error) { if (signal.aborted) throw error; - // Referenced events/artifacts are optional detail enrichment. Keep successful peers. } } }; diff --git a/gui/src/pages/models-shared.ts b/gui/src/pages/models-shared.ts index ab845a13c4..a8205616cb 100644 --- a/gui/src/pages/models-shared.ts +++ b/gui/src/pages/models-shared.ts @@ -38,8 +38,19 @@ export interface ModelRow { contextWindow?: number; contextCap?: number; contextCapped?: boolean; + /** Stored custom-row override (not the inherited ladder); only present on custom rows. */ + reasoningEfforts?: string[]; } +/** + * Reasoning-effort labels offered in the custom-model dialog. The full set of real + * `reasoning_effort` values (none, minimal, low, medium, high, xhigh, max). Deliberately + * excludes `ultra`: that is a Codex catalog label for the multi-agent collab surface, not a + * real `reasoning_effort` value — codex-rs converts it to `max` before any provider + * request, and the catalog writer appends it to every non-empty ladder anyway. + */ +export const REASONING_EFFORT_LEVELS = ["none", "minimal", "low", "medium", "high", "xhigh", "max"] as const; + export interface ProviderContextCapsResponse { cap?: number; value?: number; diff --git a/gui/src/provider-workspace/catalog.ts b/gui/src/provider-workspace/catalog.ts index 847d0ae326..fcbfac1319 100644 --- a/gui/src/provider-workspace/catalog.ts +++ b/gui/src/provider-workspace/catalog.ts @@ -97,6 +97,23 @@ function normalizedBaseUrl(value: string): string | undefined { } } +const CANONICAL_PROVIDER_PROTOCOL = new URL(CODEX_FORWARD_BASE_URL).protocol; +function providerEndpoint(host: string, ...path: string[]): string { + return `${CANONICAL_PROVIDER_PROTOCOL}//${host}/${path.join("/")}`; +} + +const STATIC_MODEL_CATALOG_TRANSPORTS: Readonly<Record<string, { adapter: string; baseUrl: string }>> = { + "cline-pass": { adapter: "openai-chat", baseUrl: providerEndpoint("api.cline.bot", "api", "v1") }, + "mimo-free": { adapter: "mimo-free", baseUrl: providerEndpoint("api.xiaomimimo.com", "api", "free-ai", "openai", "chat") }, +}; + +/** Keep the Providers toggle aligned with the backend's canonical static-catalog boundary. */ +export function providerSupportsLiveModelDiscovery(name: string, provider: WorkspaceProvider): boolean { + const canonical = STATIC_MODEL_CATALOG_TRANSPORTS[name]; + if (!canonical || provider.adapter !== canonical.adapter) return true; + return normalizedBaseUrl(provider.baseUrl) !== normalizedBaseUrl(canonical.baseUrl); +} + /** Loopback host check shared with the provider-kind classifier (WP080a). */ export function hasLoopbackBaseUrl(baseUrl: string): boolean { try { diff --git a/gui/src/styles-models-workspace.css b/gui/src/styles-models-workspace.css index 73c4901a12..e2cc78310b 100644 --- a/gui/src/styles-models-workspace.css +++ b/gui/src/styles-models-workspace.css @@ -367,6 +367,12 @@ gap: var(--space-4); } +.models-field-stack { + display: flex; + flex-direction: column; + gap: var(--space-4); +} + .models-field { display: flex; flex-direction: column; @@ -382,6 +388,18 @@ cursor: pointer; } +/* Uniform checkbox rendering in the custom-model dialog: native checkbox size is + browser-dependent (Chrome ~13px, Safari ~16px) and can even differ between rows in + some renderers, so pin an explicit size for every checkbox in the modal. 13px is the + size Chrome renders natively here; pinning it keeps the effort-step checkboxes exactly + as large as the other dialog checkboxes. */ +.modal-card .models-modality-option input[type="checkbox"] { + width: 13px; + height: 13px; + margin: 0; + flex: none; +} + /* Beat .row { gap: 10px } (defined later in styles.css). */ .row.models-model-row, .row.models-cap-row, diff --git a/gui/tests/compatibility-community-evidence.test.ts b/gui/tests/compatibility-community-evidence.test.ts new file mode 100644 index 0000000000..6605e7c64a --- /dev/null +++ b/gui/tests/compatibility-community-evidence.test.ts @@ -0,0 +1,141 @@ +import { expect, test } from "bun:test"; +import { + fetchLabPageData, + fetchVerdictDetail, + parseCommunityEvidenceContext, + type CommunityEvidenceContextDto, +} from "../src/pages/compatibility-matrix-api"; +import type { VerdictDto } from "../src/pages/compatibility-matrix-shared"; +import { + LAB_CATALOG_OVERRIDES, + labSupplement, + type LabLocale, +} from "../src/i18n/lab-translations"; + +const LOCALES = Object.keys(LAB_CATALOG_OVERRIDES) as LabLocale[]; + +function validContext(): CommunityEvidenceContextDto { + return { + trustClass: "community_untrusted_v1", + locallyVerified: false, + evidence: [ + { + trustClass: "community_untrusted_v1", + status: "cryptographically_valid", + bundleId: "a".repeat(64), + publisherKeyId: "b".repeat(64), + activeRecordCount: 3, + revokedRecordCount: 1, + }, + ], + }; +} + +function json(value: unknown): Response { + return new Response(JSON.stringify(value), { + status: 200, + headers: { "content-type": "application/json" }, + }); +} + +test("Compatibility Matrix parses only bounded quarantined community evidence context", () => { + expect(parseCommunityEvidenceContext(validContext())).toEqual(validContext()); + expect(parseCommunityEvidenceContext({ ...validContext(), locallyVerified: true })).toBeNull(); + expect(parseCommunityEvidenceContext({ ...validContext(), trustClass: "local" })).toBeNull(); + expect(parseCommunityEvidenceContext({ ...validContext(), unexpected: true })).toBeNull(); + expect(parseCommunityEvidenceContext({ + ...validContext(), + evidence: [{ ...validContext().evidence[0]!, unexpected: true }], + })).toBeNull(); + expect(parseCommunityEvidenceContext({ + ...validContext(), + evidence: [{ ...validContext().evidence[0]!, bundleId: "A".repeat(64) }], + })).toBeNull(); + expect(parseCommunityEvidenceContext({ + ...validContext(), + evidence: [{ ...validContext().evidence[0]!, publisherKeyId: "z".repeat(64) }], + })).toBeNull(); + expect(parseCommunityEvidenceContext({ + ...validContext(), + evidence: [{ ...validContext().evidence[0]!, activeRecordCount: -1 }], + })).toBeNull(); + expect(parseCommunityEvidenceContext({ + ...validContext(), + evidence: [{ ...validContext().evidence[0]!, activeRecordCount: 1.5 }], + })).toBeNull(); + expect(parseCommunityEvidenceContext({ + ...validContext(), + evidence: [{ ...validContext().evidence[0]!, status: "locally_verified" }], + })).toBeNull(); + expect(parseCommunityEvidenceContext({ + ...validContext(), + evidence: Array.from({ length: 4097 }, () => validContext().evidence[0]!), + })).toBeNull(); +}); + +test("Compatibility Matrix community copy is localized and explicitly non-authoritative", () => { + for (const locale of LOCALES) { + expect(labSupplement(locale, "community.title")).toBeTruthy(); + expect(labSupplement(locale, "community.notLocalVerdict")).toBeTruthy(); + expect(labSupplement(locale, "community.bundles")).toBeTruthy(); + expect(labSupplement(locale, "community.activeRecords")).toBeTruthy(); + expect(labSupplement(locale, "community.revokedRecords")).toBeTruthy(); + } + expect(labSupplement("en", "community.notLocalVerdict")).toMatch(/untrusted|not included|local verdict/i); +}); + +test("community evidence is fetched once as page-global context, never as verdict detail", async () => { + const originalFetch = globalThis.fetch; + const requested: string[] = []; + const context = validContext(); + globalThis.fetch = (async (input: RequestInfo | URL) => { + const url = String(input); + requested.push(url); + if (url.endsWith("/api/lab/status")) { + return json({ + projectionAvailable: true, + subjectCount: 0, + verdictCount: 0, + observationCount: 0, + eventCount: 0, + }); + } + if (url.includes("/api/lab/verdicts?")) return json({ verdicts: [], hasMore: false }); + if (url.includes("/api/lab/subjects?")) return json({ subjects: [], hasMore: false }); + if (url.endsWith("/api/lab/public/community")) return json(context); + if (url.endsWith("/api/lab/subjects/subject-alpha")) { + return json({ subject: { subjectKind: "protocol", subjectSchemaVersion: 1, inboundProtocol: "openai-chat" } }); + } + if (url.includes("/api/lab/observations?")) return json({ observations: [], hasMore: false }); + throw new Error(`unexpected optional detail request: ${url}`); + }) as typeof fetch; + + try { + const signal = new AbortController().signal; + const page = await fetchLabPageData("http://127.0.0.1:4096", {}, signal); + expect(page.community).toEqual(context); + expect(requested.filter(url => url.endsWith("/api/lab/public/community"))).toHaveLength(1); + + const verdict: VerdictDto = { + projectionKey: "v1", + subjectId: "subject-alpha", + evidenceLayer: "protocol_conformance", + suiteId: "responses-core", + suiteVersion: "1", + suiteManifestDigest: "digest-a", + projectionSpecVersion: "cl-02.v1", + verdict: "VERIFIED", + asOf: 1_700_000_000_000, + scenarioManifestDigests: [], + claimSourceDigest: null, + contributingEventIds: [], + contradictingEventIds: [], + notes: [], + }; + const detail = await fetchVerdictDetail("http://127.0.0.1:4096", verdict, signal); + expect(detail).not.toHaveProperty("community"); + expect(requested.filter(url => url.endsWith("/api/lab/public/community"))).toHaveLength(1); + } finally { + globalThis.fetch = originalFetch; + } +}); diff --git a/gui/tests/compatibility-lab-followup.test.tsx b/gui/tests/compatibility-lab-followup.test.tsx new file mode 100644 index 0000000000..79f74adb49 --- /dev/null +++ b/gui/tests/compatibility-lab-followup.test.tsx @@ -0,0 +1,196 @@ +/** @jsxImportSource react */ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { Window } from "happy-dom"; +import { act } from "react"; +import type { Root } from "react-dom/client"; +import { LanguageProvider } from "../src/i18n/provider"; +import CompatibilityMatrix from "../src/pages/CompatibilityMatrix"; +import { clearClientResourceStoresForTests } from "../src/client-resource"; + +const originalFetch = globalThis.fetch; +let restoreGlobals: (() => void) | undefined; +let previousLanguageDescriptor: PropertyDescriptor | undefined; +let testWindow: Window; + +const API_BASE = "http://127.0.0.1:4096"; + +const STATUS_AVAILABLE = { + projectionAvailable: true, + subjectCount: 1, + verdictCount: 1, + observationCount: 0, + eventCount: 2, + builtAtMs: 1_700_000_000_000, +}; + +const SUBJECTS = { + subjects: [{ subjectId: "subject-alpha", subjectKind: "protocol" }], + hasMore: false, +}; + +const SUBJECT_DETAIL = { + subject: { + subjectKind: "protocol", + subjectSchemaVersion: 1, + inboundProtocol: "openai-chat", + }, +}; + +const EVENT_DETAIL = { + event: { + eventKind: "observation", + eventId: "e1", + recordedAt: 1_700_000_000_040, + producer: "lab", + producerVersion: "1", + subjectId: "subject-alpha", + evidenceLayer: "protocol_conformance", + suiteId: "responses-core", + outcome: "pass", + excluded: false, + exclusionReason: null, + }, +}; + +function verdictPage(eventIds: string[]) { + return { + verdicts: [{ + projectionKey: "k1", + subjectId: "subject-alpha", + evidenceLayer: "protocol_conformance", + suiteId: "responses-core", + suiteVersion: "1", + suiteManifestDigest: "digest-a", + projectionSpecVersion: "cl-02.v1", + verdict: "VERIFIED", + asOf: 1_700_000_000_100, + scenarioManifestDigests: [], + claimSourceDigest: null, + contributingEventIds: eventIds, + contradictingEventIds: [], + notes: [], + }], + hasMore: true, + nextCursor: "cursor-2", + }; +} + +type FetchOptions = { + failLoadMoreOnce?: boolean; + partialEvents?: boolean; +}; + +function installLabFetch(opts: FetchOptions = {}) { + const requests: string[] = []; + let loadMoreAttempts = 0; + globalThis.fetch = (async (input: RequestInfo | URL) => { + const url = String(input); + requests.push(url); + if (url.endsWith("/api/lab/status")) return Response.json(STATUS_AVAILABLE); + if (url.includes("/api/lab/verdicts")) { + if (url.includes("cursor=cursor-2")) { + loadMoreAttempts += 1; + if (opts.failLoadMoreOnce && loadMoreAttempts === 1) { + return new Response("unavailable", { status: 503 }); + } + return Response.json({ verdicts: [], hasMore: false }); + } + return Response.json(verdictPage(opts.partialEvents ? ["e1", "missing-event"] : ["e1"])); + } + if (url.includes("/api/lab/subjects/subject-alpha")) return Response.json(SUBJECT_DETAIL); + if (url.includes("/api/lab/subjects")) return Response.json(SUBJECTS); + if (url.includes("/api/lab/observations")) return Response.json({ observations: [], hasMore: false }); + if (url.includes("/api/lab/events/e1")) return Response.json(EVENT_DETAIL); + if (url.includes("/api/lab/events/missing-event")) return new Response("gone", { status: 404 }); + if (url.includes("/api/lab/artifacts/")) return new Response("gone", { status: 404 }); + if (url.includes("/api/lab/production-signals")) return new Response("gone", { status: 404 }); + return new Response("{}", { status: 404 }); + }) as typeof fetch; + return { requests }; +} + +beforeEach(() => { + clearClientResourceStoresForTests(); + testWindow = new Window({ url: "http://localhost/#models/compatibility" }); + previousLanguageDescriptor = Object.getOwnPropertyDescriptor(globalThis.navigator, "language"); + Object.defineProperty(globalThis.navigator, "language", { configurable: true, value: "en-US" }); + const keys = ["document", "window", "localStorage", "IS_REACT_ACT_ENVIRONMENT"] as const; + const previous = Object.fromEntries( + keys.map(key => [key, Object.getOwnPropertyDescriptor(globalThis, key)]), + ) as Record<(typeof keys)[number], PropertyDescriptor | undefined>; + Object.defineProperties(globalThis, { + document: { configurable: true, value: testWindow.document }, + window: { configurable: true, value: testWindow }, + localStorage: { configurable: true, value: testWindow.localStorage }, + IS_REACT_ACT_ENVIRONMENT: { configurable: true, value: true }, + }); + restoreGlobals = () => { + for (const key of keys) { + const descriptor = previous[key]; + if (descriptor) Object.defineProperty(globalThis, key, descriptor); + else delete (globalThis as Record<string, unknown>)[key]; + } + if (previousLanguageDescriptor) { + Object.defineProperty(globalThis.navigator, "language", previousLanguageDescriptor); + } + }; +}); + +afterEach(() => { + restoreGlobals?.(); + globalThis.fetch = originalFetch; + clearClientResourceStoresForTests(); + testWindow.close(); +}); + +async function waitFor(predicate: () => boolean, timeoutMs = 3000): Promise<void> { + const start = Date.now(); + while (!predicate()) { + if (Date.now() - start > timeoutMs) throw new Error("waitFor timed out"); + await act(async () => { + await new Promise<void>(resolve => testWindow.setTimeout(resolve, 10)); + }); + } +} + +async function renderMatrix(): Promise<{ root: Root; container: HTMLDivElement }> { + const { createRoot } = await import("react-dom/client"); + const container = testWindow.document.createElement("div"); + testWindow.document.body.appendChild(container); + const root = createRoot(container); + await act(async () => { + root.render(<LanguageProvider><CompatibilityMatrix apiBase={API_BASE} /></LanguageProvider>); + }); + return { root, container }; +} + +test("load-more failures stay visible and a retry can recover", async () => { + const { requests } = installLabFetch({ failLoadMoreOnce: true }); + const { root, container } = await renderMatrix(); + await waitFor(() => container.querySelector(".lab-load-more button") !== null); + const button = container.querySelector(".lab-load-more button") as HTMLButtonElement; + await act(async () => { button.click(); }); + await waitFor(() => container.querySelector(".notice-err")?.textContent?.includes(String(503)) ?? false); + expect(container.textContent).toContain("Verified"); + expect(container.querySelector(".lab-load-more button")).not.toBeNull(); + + await act(async () => { + (container.querySelector(".lab-load-more button") as HTMLButtonElement).click(); + }); + await waitFor(() => container.querySelector(".notice-err") === null); + await waitFor(() => container.querySelector(".lab-load-more button") === null); + expect(requests.filter(url => url.includes("/api/lab/verdicts") && url.includes("cursor=cursor-2"))).toHaveLength(2); + expect(container.textContent).toContain("Verified"); + await act(async () => root.unmount()); +}); + +test("verdict detail reports when referenced evidence events are only partially available", async () => { + installLabFetch({ partialEvents: true }); + const { root, container } = await renderMatrix(); + await waitFor(() => container.querySelector('button[data-verdict-detail="k1"]') !== null); + const button = container.querySelector('button[data-verdict-detail="k1"]') as HTMLButtonElement; + await act(async () => { button.click(); }); + await waitFor(() => container.querySelector(".lab-detail-pane")?.textContent?.includes("Evidence events (1/2)") ?? false); + expect(container.querySelector(".lab-detail-pane")?.textContent).toContain("Evidence events (1/2)"); + await act(async () => root.unmount()); +}); diff --git a/gui/tests/provider-settings-live-models-provenance.test.tsx b/gui/tests/provider-settings-live-models-provenance.test.tsx index 097780177e..f40ee8067a 100644 --- a/gui/tests/provider-settings-live-models-provenance.test.tsx +++ b/gui/tests/provider-settings-live-models-provenance.test.tsx @@ -115,3 +115,46 @@ test("changing an explicit false to true sends an explicit liveModels choice", a expect(patches[0]?.liveModels).toBe(true); await act(async () => { root.unmount(); }); }); + +test("canonical ClinePass shows the static catalog as disabled even with stale liveModels true", async () => { + const { root, container } = await mountSettings({ + name: "cline-pass", + adapter: "openai-chat", + baseUrl: "https://api.cline.bot/api/v1", + authMode: "key", + liveModels: true, + } as WorkspaceItem); + const toggles = container.querySelectorAll<HTMLInputElement>('input[type="checkbox"]'); + + expect(toggles[1]?.disabled).toBe(true); + expect(toggles[1]?.checked).toBe(false); + await act(async () => { root.unmount(); }); +}); + +test("same-named custom MiMo provider keeps live discovery editable", async () => { + const { root, container } = await mountSettings({ + name: "mimo-free", + adapter: "openai-chat", + baseUrl: "https://example.test/v1", + authMode: "key", + liveModels: true, + } as WorkspaceItem); + const toggles = container.querySelectorAll<HTMLInputElement>('input[type="checkbox"]'); + + expect(toggles[1]?.disabled).toBe(false); + expect(toggles[1]?.checked).toBe(true); + await act(async () => { root.unmount(); }); +}); + +test("key-optional auth fallback remains local for unrelated providers", async () => { + const { root, container } = await mountSettings({ + name: "custom-provider", + adapter: "openai-chat", + baseUrl: "https://example.test/v1", + keyOptional: true, + } as WorkspaceItem); + const selects = container.querySelectorAll<HTMLSelectElement>("select.input"); + + expect(selects[1]?.value).toBe("local"); + await act(async () => { root.unmount(); }); +}); diff --git a/src/adapters/mimo-free.ts b/src/adapters/mimo-free.ts index b02ea0d20e..a257e7d4c9 100644 --- a/src/adapters/mimo-free.ts +++ b/src/adapters/mimo-free.ts @@ -40,6 +40,18 @@ function randomUserAgent(): string { return USER_AGENTS[Math.floor(Math.random() * USER_AGENTS.length)]!; } +function isCanonicalMimoFreeEndpoint(baseUrl: string): boolean { + try { + const actual = new URL(baseUrl.trim()); + const expected = new URL(MIMO_CHAT_URL); + actual.pathname = actual.pathname.replace(/\/+$/, "") || "/"; + expected.pathname = expected.pathname.replace(/\/+$/, "") || "/"; + return actual.toString().replace(/\/$/, "") === expected.toString().replace(/\/$/, ""); + } catch { + return false; + } +} + /** * Anonymous per-install client id for the bootstrap `client` field. A random UUID * persisted under the config dir (OPENCODEX_HOME-aware) — deliberately NOT derived @@ -194,6 +206,11 @@ export function injectMimoSystemMarker(body: unknown): unknown { * On 401/403, flushes the JWT cache and retries once via fetchResponse. */ export function createMimoFreeAdapter(provider: OcxProviderConfig): ProviderAdapter { + if (!isCanonicalMimoFreeEndpoint(provider.baseUrl)) { + throw new Error( + "The mimo-free adapter only supports the canonical Xiaomi MiMo Free endpoint. Use openai-chat for a custom endpoint.", + ); + } const base = createOpenAIChatAdapter(provider); // Per-adapter session-affinity id (random, per process instance). const sessionId = `ses_${Math.random().toString(36).slice(2, 26)}`; diff --git a/src/adapters/registry.ts b/src/adapters/registry.ts new file mode 100644 index 0000000000..2b88338b3e --- /dev/null +++ b/src/adapters/registry.ts @@ -0,0 +1,144 @@ +import { createAnthropicAdapter } from "./anthropic"; +import { createAzureAdapter } from "./azure"; +import type { ProviderAdapter } from "./base"; +import { createCommandCodeAdapter } from "./command-code"; +import { createCursorAdapter } from "./cursor"; +import { createGoogleAdapter } from "./google"; +import { createKiroAdapter } from "./kiro"; +import { createMimoFreeAdapter } from "./mimo-free"; +import { createOpenAIChatAdapter } from "./openai-chat"; +import { createResponsesPassthroughAdapter } from "./openai-responses"; +import type { OcxProviderConfig } from "../types"; + +export type AdapterCacheRetention = "none" | "short" | "long"; + +export interface AdapterFactoryContext { + cacheRetention?: AdapterCacheRetention; +} + +export type AdapterWire = + | "command-code" + | "openai-chat" + | "anthropic" + | "openai-responses" + | "google" + | "kiro" + | "cursor"; + +export type AdapterMutationContract = + | "codex-owned" + | "codex-owned-with-gated-native-fallback"; + +type AdapterFactory = ( + provider: OcxProviderConfig, + context: AdapterFactoryContext, +) => ProviderAdapter; + +type DirectAdapterDefinition = { + wire: AdapterWire; + mutation: AdapterMutationContract; + create: AdapterFactory; +}; + +type InheritedAdapterDefinition = { + /** Semantic contract inheritance only. Runtime construction remains independent. */ + contractParent: string; + create: AdapterFactory; +}; + +type AdapterDefinition = DirectAdapterDefinition | InheritedAdapterDefinition; + +export const ADAPTER_REGISTRY = { + "command-code": { + wire: "command-code", + mutation: "codex-owned", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => createCommandCodeAdapter(provider), + }, + "openai-chat": { + wire: "openai-chat", + mutation: "codex-owned", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => createOpenAIChatAdapter(provider), + }, + anthropic: { + wire: "anthropic", + mutation: "codex-owned", + create: (provider: OcxProviderConfig, context: AdapterFactoryContext) => + createAnthropicAdapter(provider, context.cacheRetention), + }, + "openai-responses": { + wire: "openai-responses", + mutation: "codex-owned", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => + createResponsesPassthroughAdapter(provider), + }, + google: { + wire: "google", + mutation: "codex-owned", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => createGoogleAdapter(provider), + }, + kiro: { + wire: "kiro", + mutation: "codex-owned", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => createKiroAdapter(provider), + }, + azure: { + contractParent: "openai-responses", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => createAzureAdapter(provider), + }, + "azure-openai": { + contractParent: "openai-responses", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => createAzureAdapter(provider), + }, + cursor: { + wire: "cursor", + mutation: "codex-owned-with-gated-native-fallback", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => createCursorAdapter(provider), + }, + "mimo-free": { + contractParent: "openai-chat", + create: (provider: OcxProviderConfig, _context: AdapterFactoryContext) => createMimoFreeAdapter(provider), + }, +} as const satisfies Record<string, AdapterDefinition>; + +export type AdapterId = keyof typeof ADAPTER_REGISTRY; +export type RegisteredAdapterDefinition = typeof ADAPTER_REGISTRY[AdapterId]; + +export function adapterDefinitions(): Array<[AdapterId, RegisteredAdapterDefinition]> { + return Object.entries(ADAPTER_REGISTRY) as Array<[AdapterId, RegisteredAdapterDefinition]>; +} + +export function getAdapterDefinition(adapterId: unknown): RegisteredAdapterDefinition | undefined { + if (typeof adapterId !== "string" || !Object.hasOwn(ADAPTER_REGISTRY, adapterId)) return undefined; + return ADAPTER_REGISTRY[adapterId as AdapterId]; +} + +export function effectiveAdapterContract(adapterId: string): Readonly<{ + wire: AdapterWire; + mutation: AdapterMutationContract; +}> { + const visited = new Set<string>(); + let current = adapterId; + + while (true) { + if (visited.has(current)) { + throw new Error(`Adapter contract cycle detected at ${current}`); + } + visited.add(current); + + const definition = getAdapterDefinition(current); + if (!definition) throw new Error(`Unknown adapter: ${current}`); + if ("wire" in definition) { + return { wire: definition.wire, mutation: definition.mutation }; + } + current = definition.contractParent; + } +} + +export function createRegisteredAdapter( + provider: OcxProviderConfig, + context: AdapterFactoryContext = {}, +): ProviderAdapter { + const definition = getAdapterDefinition(provider.adapter); + if (!definition) throw new Error(`Unknown adapter: ${provider.adapter}`); + return definition.create(provider, context); +} diff --git a/src/cli/lab.ts b/src/cli/lab.ts index 013b96327a..81dcf55745 100644 --- a/src/cli/lab.ts +++ b/src/cli/lab.ts @@ -36,6 +36,7 @@ import { queryPassiveProductionSignals, type PassiveProductionQueryResultV1, } from "../lab/query"; +import { isLabRouteSubjectId } from "../usage/log"; import { CliUsageError, RuntimeApiError, @@ -63,6 +64,14 @@ import { planManualLabRun } from "../lab/automation/planner"; import { listLabAutomationRuns } from "../lab/automation/runs-query"; import { LabAutomationError, type LabAutomationLayer } from "../lab/automation/types"; import { createProductionLabRouteExecutor } from "../lib/lab-live-route-production"; +import { + exportLocalPublicEvidence, + importCommunityEvidenceFile, + listCommunityEvidenceContext, + previewLocalPublicEvidence, + verifyPublicEvidenceFile, + type PublicVerificationSummaryV1, +} from "../lab/public"; const USAGE = `Usage: ocx lab status [--json] @@ -76,6 +85,11 @@ const USAGE = `Usage: ocx lab artifacts [--status <s>] [--artifact-class <c>] [--limit <n>] [--cursor <c>] [--json] ocx lab artifact <digest> [--json] ocx lab catalog [--layer <layer>] [--suite <id>] [--json] + ocx lab public preview --event <eventId> [--event <eventId> ...] [--json] + ocx lab public export --event <eventId> [--event <eventId> ...] [--json] + ocx lab public verify --file <bundle.json> [--json] + ocx lab public import --file <bundle.json> [--json] + ocx lab public community [--json] ocx lab automation status [--json] ocx lab automation enable [--protocol] [--live] [--json] ocx lab automation disable [--json] @@ -223,6 +237,119 @@ function runListLines(page: ReturnType<typeof listLabAutomationRuns>): string[] return lines.length > 0 ? lines : ["No automation runs"]; } +function takeRepeatedOptions(args: string[], flag: string): string[] { + const values: string[] = []; + while (true) { + const index = args.indexOf(flag); + if (index < 0) break; + const value = args[index + 1]; + if (!value || value.startsWith("--")) { + throw new CliUsageError(`${flag} requires a value`, USAGE); + } + values.push(value); + args.splice(index, 2); + } + return values; +} + +function publicPreviewLines(result: ReturnType<typeof previewLocalPublicEvidence>): string[] { + return [ + `Public evidence preview: ${result.bundle.records.length} exportable record(s)`, + `Excluded: ${result.excluded.length}`, + "Unsigned local preview; no publisher key or remote publish is created.", + ]; +} + +function publicExportLines(result: ReturnType<typeof exportLocalPublicEvidence>): string[] { + return [ + "Public evidence exported locally", + `Bundle: ${result.bundle.bundleId}`, + `Publisher: ${result.bundle.publisher.keyId}`, + `Path: ${result.stored.path}`, + `Excluded: ${result.excluded.length}`, + "No remote publish occurred.", + ]; +} + +function publicVerificationLines(result: PublicVerificationSummaryV1): string[] { + if (result.status !== "cryptographically_valid") { + return [ + `Public evidence verification: ${result.status}`, + "Not locally verified.", + ]; + } + return [ + "Public evidence verification: cryptographically valid", + `Bundle: ${result.bundleId}`, + `Publisher: ${result.publisherKeyId}`, + "Not locally verified. Signature validity proves integrity/continuity only.", + ]; +} + +function handlePublicLabCommand( + argv: string[], + wantsJson: boolean, + configDir: string, +): void { + const [action, ...restInput] = argv; + const rest = [...restInput]; + switch (action) { + case "preview": { + const eventIds = takeRepeatedOptions(rest, "--event"); + rejectArgs(rest, USAGE); + const result = previewLocalPublicEvidence({ eventIds }, configDir); + printData(result, wantsJson, publicPreviewLines(result)); + return; + } + case "export": { + const eventIds = takeRepeatedOptions(rest, "--event"); + rejectArgs(rest, USAGE); + const result = exportLocalPublicEvidence({ eventIds }, configDir); + printData(result, wantsJson, publicExportLines(result)); + return; + } + case "verify": { + const path = takeOption(rest, "--file"); + if (!path) throw new CliUsageError("public verify requires --file", USAGE); + rejectArgs(rest, USAGE); + const result = verifyPublicEvidenceFile(path); + printData(result, wantsJson, publicVerificationLines(result)); + if (result.status !== "cryptographically_valid") { + throw new RuntimeApiError(`public evidence verification failed: ${result.status}`, 422, result); + } + return; + } + case "import": { + const path = takeOption(rest, "--file"); + if (!path) throw new CliUsageError("public import requires --file", USAGE); + rejectArgs(rest, USAGE); + const result = importCommunityEvidenceFile(path, configDir); + printData(result, wantsJson, [ + `Community evidence imported: ${result.bundleId}`, + `Publisher: ${result.publisherKeyId}`, + "Trust: community_untrusted_v1; not locally verified.", + ]); + return; + } + case "community": { + rejectArgs(rest, USAGE); + const result = listCommunityEvidenceContext(configDir); + const lines = result.evidence.length > 0 + ? result.evidence.map((row) => + `${row.bundleId} publisher=${row.publisherKeyId} active=${row.activeRecordCount} revoked=${row.revokedRecordCount}`, + ) + : ["No community evidence"]; + printData(result, wantsJson, [ + "Community evidence (untrusted, read-only context; not locally verified)", + ...lines, + ]); + return; + } + default: + throw new CliUsageError("unknown public subcommand", USAGE); + } +} + export async function handleLabCommand(argv: string[], deps: LabCliDeps = {}): Promise<number> { return runCliAction(async () => { const configDir = deps.configDir ?? getConfigDir(); @@ -232,6 +359,10 @@ export async function handleLabCommand(argv: string[], deps: LabCliDeps = {}): P try { switch (sub) { + case "public": { + handlePublicLabCommand(rest, wantsJson, configDir); + return; + } case "status": { rejectArgs(rest, USAGE); const status = queryLabStatus(configDir); @@ -243,6 +374,9 @@ export async function handleLabCommand(argv: string[], deps: LabCliDeps = {}): P const limit = takeIntegerOption(rest, "--limit", { min: 1 }); rejectArgs(rest, USAGE); if (!subjectId) throw new CliUsageError("--subject is required", USAGE); + if (!isLabRouteSubjectId(subjectId)) { + throw new CliUsageError("--subject must be an exact Lab route subject id", USAGE); + } const result = queryPassiveProductionSignals(subjectId, limit, configDir); printData(result, wantsJson, passiveProductionLines(result)); return; @@ -461,7 +595,7 @@ export async function handleLabCommand(argv: string[], deps: LabCliDeps = {}): P throw new CliUsageError(`unknown lab subcommand: ${sub}`, USAGE); } } catch (err) { - if (err instanceof CliUsageError) throw err; + if (err instanceof CliUsageError || err instanceof RuntimeApiError) throw err; if (err instanceof LabProjectionUnavailableError || err instanceof LabProjectionIncompatibleError) { throw new LabStateError(labErrorMessage(err)); } diff --git a/src/cli/models-runtime.ts b/src/cli/models-runtime.ts index e451bb1005..c309274c8b 100644 --- a/src/cli/models-runtime.ts +++ b/src/cli/models-runtime.ts @@ -16,7 +16,9 @@ import { const USAGE = `Usage: ocx models live [--provider <name>] [--json] ocx models edit <custom-id> [--model-id <id>] [--display-name <name|->] - [--context-window <tokens|0>] [--modalities <text,image,audio|->] [--json] + [--context-window <tokens|0>] [--modalities <text,image,audio|->] + [--reasoning-efforts <none,minimal,low,medium,high,xhigh,max,ultra|->] + [--default-reasoning-effort <level|->] [--json] ocx models <enable|disable> <provider/model|native-model> [--native] [--json] ocx models provider <name> <on|off> [--json] ocx models selected <provider> [--set <id,id...>|--clear] [--json] @@ -57,6 +59,8 @@ async function edit(argv: string[], deps: RuntimeApiDeps): Promise<void> { const displayName = takeOption(args, "--display-name"); const contextRaw = takeOption(args, "--context-window"); const modalitiesRaw = takeOption(args, "--modalities"); + const reasoningEffortsRaw = takeOption(args, "--reasoning-efforts"); + const defaultEffortRaw = takeOption(args, "--default-reasoning-effort"); rejectArgs(args, USAGE); if (modelId !== undefined) patch.modelId = modelId; if (displayName !== undefined) patch.displayName = displayName === "-" ? "" : displayName; @@ -66,6 +70,23 @@ async function edit(argv: string[], deps: RuntimeApiDeps): Promise<void> { patch.contextWindow = value === 0 ? null : value; } if (modalitiesRaw !== undefined) patch.inputModalities = modalitiesRaw === "-" ? [] : csv(modalitiesRaw); + // "-" restores inheritance by clearing the stored ladder (null); "" stores an explicit + // empty ladder (the "no reasoning" override, same as the dashboard's uncheck-all). + // Embedded blank CSV members (`low,,high`, `,,`) are malformed and must be rejected, not + // silently normalized by csv(). + if (reasoningEffortsRaw !== undefined) { + if (reasoningEffortsRaw === "-") { + patch.reasoningEfforts = null; + } else { + const trimmed = reasoningEffortsRaw.trim(); + const values = trimmed === "" ? [] : trimmed.split(",").map(value => value.trim()); + if (values.some(value => value === "")) { + throw new CliUsageError("--reasoning-efforts must be comma-separated values from none, minimal, low, medium, high, xhigh, max, ultra (\"\" for no reasoning, \"-\" to inherit)", USAGE); + } + patch.reasoningEfforts = values; + } + } + if (defaultEffortRaw !== undefined) patch.defaultReasoningEffort = defaultEffortRaw === "-" ? null : defaultEffortRaw; if (Object.keys(patch).length === 0) throw new CliUsageError("at least one edit option is required", USAGE); const result = await runtimeRequest(`/api/custom-models/${encodeURIComponent(id)}`, { method: "PUT", diff --git a/src/cli/models.ts b/src/cli/models.ts index 11787e9bcc..83403c6a01 100644 --- a/src/cli/models.ts +++ b/src/cli/models.ts @@ -5,15 +5,71 @@ import { randomUUID } from "node:crypto"; import { createInterface } from "node:readline/promises"; import { syncModelsToCodex } from "../codex/sync"; import { hasOwnProvider, isValidProviderName, loadConfig, saveConfig } from "../config"; +import { canonicalizeReasoningEfforts, isDeclaredReasoningEffort } from "../reasoning-effort"; import { routedSlug } from "../providers/slug-codec"; import { findLiveProxy } from "../server/proxy-liveness"; import type { OcxConfig, OcxCustomModel } from "../types"; -const ADD_USAGE = "Usage: ocx models add <provider> <modelId> [--display-name <name>] [--context-window <tokens>] [--modalities text,image,audio]"; +const ADD_USAGE = "Usage: ocx models add <provider> <modelId> [--display-name <name>] [--context-window <tokens>] [--modalities text,image,audio] [--reasoning-efforts <none,minimal,low,medium,high,xhigh,max,ultra>] [--default-reasoning-effort <level>]"; const REMOVE_USAGE = "Usage: ocx models remove <customId|provider/modelId> [--yes]"; const LIST_CUSTOM_USAGE = "Usage: ocx models list-custom [--json]"; const ALLOWED_MODALITIES = new Set(["text", "image", "audio"]); +/** + * Parse and validate the reasoning flags shared by `ocx models add` (offline path). + * "-" means "inherit" and omits the field entirely; "" means an explicit empty ladder + * ("no reasoning" override, the same state the dashboard stores for the toggle-off + * checkbox set). Malformed CSV like `low,,high` or `,,` is rejected instead of being + * silently normalized. Values are canonicalized into Codex ladder order so the stored + * config matches what the API stores. + */ +export function parseReasoningArgs( + reasoningEffortsValue: string | undefined, + defaultEffortValue: string | undefined, +): { reasoningEfforts?: string[]; defaultReasoningEffort?: string; error?: string } { + if (reasoningEffortsValue === undefined && defaultEffortValue === undefined) return {}; + let reasoningEfforts: string[] | undefined; + if (reasoningEffortsValue !== undefined) { + const trimmed = reasoningEffortsValue.trim(); + if (trimmed === "-") { + reasoningEfforts = undefined; + } else if (trimmed === "") { + // Explicit no-reasoning override, exactly like the API's [] / the dashboard's + // uncheck-all state. + reasoningEfforts = []; + } else { + const parts = trimmed.split(",").map(value => value.trim()); + if (parts.some(part => part === "")) { + return { error: "--reasoning-efforts must be comma-separated values from none, minimal, low, medium, high, xhigh, max, ultra (\"\" for no reasoning, \"-\" to inherit)" }; + } + const invalid = parts.filter(value => !isDeclaredReasoningEffort(value)); + if (invalid.length > 0) { + return { error: `unsupported reasoning effort: ${invalid.join(", ")} (allowed: none, minimal, low, medium, high, xhigh, max, ultra)` }; + } + reasoningEfforts = canonicalizeReasoningEfforts(parts); + } + } + let defaultReasoningEffort: string | undefined; + if (defaultEffortValue !== undefined) { + const trimmed = defaultEffortValue.trim(); + if (trimmed === "-") { + defaultReasoningEffort = undefined; + } else { + if (!isDeclaredReasoningEffort(trimmed)) { + return { error: `unsupported reasoning effort: ${trimmed} (allowed: none, minimal, low, medium, high, xhigh, max, ultra)` }; + } + if (!reasoningEfforts || reasoningEfforts.length === 0) { + return { error: "--default-reasoning-effort requires --reasoning-efforts" }; + } + if (!reasoningEfforts.includes(trimmed)) { + return { error: `--default-reasoning-effort "${trimmed}" is not in the declared reasoning efforts` }; + } + defaultReasoningEffort = trimmed; + } + } + return { reasoningEfforts, defaultReasoningEffort }; +} + interface ModelEntry { provider: string; model: string; @@ -118,6 +174,8 @@ async function handleCustomAdd(args: string[]): Promise<void> { const displayNameValue = consumeFlagValue(rest, "--display-name"); const contextWindowValue = consumeFlagValue(rest, "--context-window"); const modalitiesValue = consumeFlagValue(rest, "--modalities"); + const reasoningEffortsValue = consumeFlagValue(rest, "--reasoning-efforts"); + const defaultEffortValue = consumeFlagValue(rest, "--default-reasoning-effort"); rejectUnexpectedArgs(rest, ADD_USAGE); if (!provider || !modelId) fail("provider and modelId are required", ADD_USAGE); @@ -150,6 +208,9 @@ async function handleCustomAdd(args: string[]): Promise<void> { inputModalities = [...new Set(inputModalities)]; } + const parsed = parseReasoningArgs(reasoningEffortsValue, defaultEffortValue); + if (parsed.error) fail(parsed.error); + const existing = config.customModels ?? []; const slug = routedSlug(provider, modelId); if (existing.some(model => routedSlug(model.provider, model.modelId) === slug)) { @@ -163,6 +224,8 @@ async function handleCustomAdd(args: string[]): Promise<void> { ...(displayName ? { displayName } : {}), ...(contextWindow ? { contextWindow } : {}), ...(inputModalities ? { inputModalities } : {}), + ...(parsed.reasoningEfforts ? { reasoningEfforts: parsed.reasoningEfforts } : {}), + ...(parsed.defaultReasoningEffort ? { defaultReasoningEffort: parsed.defaultReasoningEffort } : {}), addedAt: new Date().toISOString(), }; config.customModels = [...existing, entry]; @@ -218,12 +281,14 @@ function customModelCells(model: OcxCustomModel): string[] { model.displayName ?? "-", model.contextWindow ? `${Math.round(model.contextWindow / 1000)}k` : "-", model.inputModalities?.join(",") ?? "-", + model.reasoningEfforts?.join(",") ?? "-", + model.defaultReasoningEffort ?? "-", ]; } function printCustomModelGroup(provider: string, models: OcxCustomModel[]): void { const rows = models.map(customModelCells); - const headers = ["ID", "MODEL", "DISPLAY NAME", "CONTEXT", "MODALITIES"]; + const headers = ["ID", "MODEL", "DISPLAY NAME", "CONTEXT", "MODALITIES", "EFFORTS", "DEFAULT EFFORT"]; const widths = headers.map((header, column) => Math.max(header.length, ...rows.map(row => row[column].length))); const line = (cells: string[]) => cells.map((cell, column) => cell.padEnd(widths[column])).join(" "); console.log(`${provider}:`); diff --git a/src/clients/config-export.ts b/src/clients/config-export.ts index 44240c9ef5..4c903fa0e6 100644 --- a/src/clients/config-export.ts +++ b/src/clients/config-export.ts @@ -664,6 +664,15 @@ export interface PiModelEntry { input: string[]; contextWindow?: number; maxTokens?: number; + /** Advertised when the catalog row carries a non-empty effort ladder. */ + reasoning?: true; + /** + * Constrains pi's own level scale (minimal..max) to the declared ladder: members map to + * themselves, everything else is hidden (`null`). Without it pi would offer levels the + * ladder does not contain — harmless for provider-config ladders (the proxy clamps those + * at the wire) but a real 400 risk for custom-row ladders, which are advertisement-only. + */ + thinkingLevelMap?: Record<string, string | null>; } export interface PiProviderBlock { @@ -836,16 +845,24 @@ export interface DshGeneratedConfig { * Pi's `~/.pi/agent/models.json` shape. `models` is an ARRAY (identity lives in `id`), * unlike OpenCode's keyed object. * - * Two fields are deliberately absent. `cost` requires all four price fields and we have - * no price data at all, so emitting zeros would assert every routed model is free. - * `reasoning` is a boolean in Pi while our catalog carries an effort list — mapping one - * to the other would be a guess. + * Two fields were deliberately absent once. `cost` still is: it requires all four price + * fields and we have no price data at all, so emitting zeros would assert every routed + * model is free. `reasoning` used to be omitted because Pi's boolean and the catalog's + * effort ladder did not obviously map — but a NON-EMPTY ladder is the catalog's own + * statement that the model accepts reasoning parameters (adapters honor `reasoning_effort`), + * and an empty or absent ladder is the statement that it does not. Emitting `reasoning: + * true` exactly for rows with a ladder is therefore not a guess; it is what makes Pi's + * effort control appear for routed models at all. The export also emits a `thinkingLevelMap` + * that hides every pi level outside the declared ladder, so pi never offers (and sends) an + * effort the ladder does not contain — custom-row ladders are catalog advertisement only + * and get no wire clamp, so this map is what keeps pi honest for those. Users who need a + * different mapping can still hand-tune `thinkingLevelMap` afterwards. * * Pi's input enum IS verified: its documented model configuration accepts only * `text` and `image`, and a validation failure yields an EMPTY model config * rather than dropping the offending entry — one bad value costs every routed - * model. The rest of this contract (omitting `cost` and `reasoning`) is still - * ours rather than a claim about Pi's acceptance. + * model. The rest of this contract (omitting `cost`) is still ours rather than + * a claim about Pi's acceptance. */ function buildPiClientConfig(ctx: ExportContext): PiGeneratedConfig { const models: PiModelEntry[] = []; @@ -862,6 +879,21 @@ function buildPiClientConfig(ctx: ExportContext): PiGeneratedConfig { name: exportModelLabel(model), input, }; + if (Array.isArray(model.reasoningEfforts) && model.reasoningEfforts.length > 0) { + entry.reasoning = true; + const efforts = model.reasoningEfforts; + entry.thinkingLevelMap = { + // pi's off level maps to the declared `none` sentinel (the proxy omits the + // reasoning parameter for it); hidden when the ladder does not declare none. + off: efforts.includes("none") ? "none" : null, + minimal: efforts.includes("minimal") ? "minimal" : null, + low: efforts.includes("low") ? "low" : null, + medium: efforts.includes("medium") ? "medium" : null, + high: efforts.includes("high") ? "high" : null, + xhigh: efforts.includes("xhigh") ? "xhigh" : null, + max: efforts.includes("max") ? "max" : efforts.includes("ultra") ? "ultra" : null, + }; + } const context = authoritativeContextWindow(model.contextWindow); if (context !== undefined) { entry.contextWindow = context; diff --git a/src/codex/catalog/effort.ts b/src/codex/catalog/effort.ts index b1ac6b540d..4e1c9fa7fe 100644 --- a/src/codex/catalog/effort.ts +++ b/src/codex/catalog/effort.ts @@ -153,8 +153,9 @@ export function applyReasoningLevels( // (no ultra->max client conversion) and codex-rs validates it by catalog membership, // so a missing max rung hard-fails spawn_agent effort overrides. The wire stays honest: // routed adapters clamp via clampToSupportedCodexEffort and natives via - // nativeEffortClamp (max -> the model's real top rung). - if (!preserveExact && efforts.length > 0) { + // nativeEffortClamp (max -> the model's real top rung). A `none`-only ladder is NOT + // reasoning-capable, so it must not grow synthetic top rungs. + if (!preserveExact && efforts.length > 0 && efforts.some(effort => effort !== "none" && effort !== "minimal")) { const additions: string[] = []; if (!efforts.includes("max")) additions.push("max"); if (!efforts.includes("ultra")) additions.push("ultra"); @@ -177,7 +178,9 @@ export function applyReasoningLevels( } entry.default_reasoning_level = defaultOverride && efforts.includes(defaultOverride) ? defaultOverride - : efforts.includes("medium") ? "medium" : efforts.includes("high") ? "high" : efforts[0]; + : efforts.includes("medium") ? "medium" : efforts.includes("high") ? "high" + // Sentinels never become the implicit default when real rungs are declared. + : efforts.find(effort => effort !== "none" && effort !== "minimal") ?? efforts[0]; } export function isGpt56NativeSlug(slug: string): boolean { diff --git a/src/codex/catalog/provider-fetch.ts b/src/codex/catalog/provider-fetch.ts index 137e8e441a..f617f23022 100644 --- a/src/codex/catalog/provider-fetch.ts +++ b/src/codex/catalog/provider-fetch.ts @@ -1775,14 +1775,26 @@ async function gatherRoutedModelsUncached( ? { inputModalities: cm.inputModalities } : codexForwardNativeCapabilityAlias ? { inputModalities: nativeInputModalities(cm.modelId) } : {}), ...(typeof supportsReasoningSummaries === "boolean" ? { supportsReasoningSummaries } : {}), + // Native-alias defaults apply only where the custom row declares nothing: the explicit + // spreads below must win (later in object order), so a stored `[]` stays empty and a + // declared ladder is never replaced by the alias's native ladder. ...(codexForwardNativeCapabilityAlias ? { codexForwardNativeCapabilityAlias: true, - reasoningEfforts: nativeReasoningEfforts(cm.modelId), parallelToolCalls: nativeParallelToolCalls(cm.modelId), - ...(nativeAliasDefaultEffort ? { defaultReasoningEffort: nativeAliasDefaultEffort } : {}), + ...(Array.isArray(cm.reasoningEfforts) + ? {} + : { + reasoningEfforts: nativeReasoningEfforts(cm.modelId), + ...(nativeAliasDefaultEffort ? { defaultReasoningEffort: nativeAliasDefaultEffort } : {}), + }), } : {}), + // Explicit custom-row ladder wins over the inherited provider row below: the merge only + // gap-fills, so a stored `[]` (explicit "no reasoning") or a declared ladder is kept + // verbatim instead of being replaced by the replaced row's metadata. + ...(Array.isArray(cm.reasoningEfforts) ? { reasoningEfforts: [...cm.reasoningEfforts] } : {}), + ...(cm.defaultReasoningEffort ? { defaultReasoningEffort: cm.defaultReasoningEffort } : {}), }; // #962: the dedupe below drops the provider-derived row this custom row replaces. Inherit that // row's provider capability metadata (reasoning ladder, default effort, parallel tool calls, @@ -1791,13 +1803,19 @@ async function gatherRoutedModelsUncached( // noReasoningModels model loses its empty ladder and the catalog synthesizes the generic one, // which Codex then rejects for spawn_agent with effort "none". const replaced = replacedByRoutedSlug.get(routedSlug(cm.provider, cm.modelId)); + // The final ladder is what the catalog will advertise; the inherited default only rides + // along when it is actually a member — otherwise a provider default like "xhigh" would + // re-apply onto a narrower custom ladder and override the fallback in applyReasoningLevels. + const effectiveLadder = base.reasoningEfforts ?? replaced?.reasoningEfforts; const merged: CatalogModel = replaced ? { ...base, ...(base.contextWindow === undefined && replaced.contextWindow !== undefined ? { contextWindow: replaced.contextWindow } : {}), ...(base.maxInputTokens === undefined && replaced.maxInputTokens !== undefined ? { maxInputTokens: replaced.maxInputTokens } : {}), ...(base.inputModalities === undefined && replaced.inputModalities !== undefined ? { inputModalities: replaced.inputModalities } : {}), ...(base.reasoningEfforts === undefined && replaced.reasoningEfforts !== undefined ? { reasoningEfforts: replaced.reasoningEfforts } : {}), - ...(base.defaultReasoningEffort === undefined && replaced.defaultReasoningEffort !== undefined ? { defaultReasoningEffort: replaced.defaultReasoningEffort } : {}), + ...(base.defaultReasoningEffort === undefined && replaced.defaultReasoningEffort !== undefined + && Array.isArray(effectiveLadder) && effectiveLadder.includes(replaced.defaultReasoningEffort) + ? { defaultReasoningEffort: replaced.defaultReasoningEffort } : {}), ...(base.parallelToolCalls === undefined && replaced.parallelToolCalls !== undefined ? { parallelToolCalls: replaced.parallelToolCalls } : {}), ...(base.supportsVerbosity === undefined && replaced.supportsVerbosity !== undefined ? { supportsVerbosity: replaced.supportsVerbosity } : {}), ...(base.supportsReasoningSummaries === undefined && replaced.supportsReasoningSummaries !== undefined ? { supportsReasoningSummaries: replaced.supportsReasoningSummaries } : {}), diff --git a/src/lab/conformance/jcs.ts b/src/lab/conformance/jcs.ts index 6bbcb923c7..eaf30b6219 100644 --- a/src/lab/conformance/jcs.ts +++ b/src/lab/conformance/jcs.ts @@ -1,5 +1,40 @@ /** RFC 8785 JSON Canonicalization Scheme (JCS) for deterministic equality. */ +function assertValidUnicodeScalarString(value: string): void { + for (let index = 0; index < value.length; index += 1) { + const code = value.charCodeAt(index); + if (code >= 0xd800 && code <= 0xdbff) { + const next = value.charCodeAt(index + 1); + if (!(next >= 0xdc00 && next <= 0xdfff)) { + throw new TypeError("jcsStringify: lone UTF-16 surrogate is not valid Unicode"); + } + index += 1; + continue; + } + if (code >= 0xdc00 && code <= 0xdfff) { + throw new TypeError("jcsStringify: lone UTF-16 surrogate is not valid Unicode"); + } + } +} + +function stringifyJcsString(value: string): string { + assertValidUnicodeScalarString(value); + return JSON.stringify(value); +} + +function assertDenseJsonArray(value: readonly unknown[]): void { + for (let index = 0; index < value.length; index += 1) { + if (!Object.prototype.hasOwnProperty.call(value, index)) { + throw new TypeError("jcsStringify: sparse arrays / array holes are not representable in JCS"); + } + } + for (const key of Object.keys(value)) { + if (!/^(?:0|[1-9]\d*)$/.test(key) || Number(key) >= value.length) { + throw new TypeError("jcsStringify: arrays with extra enumerable properties are not representable in JCS"); + } + } +} + export function jcsStringify(value: unknown): string { if (value === undefined) throw new TypeError("jcsStringify: undefined is not representable in JCS"); if (value === null || typeof value === "boolean") return JSON.stringify(value); @@ -7,14 +42,19 @@ export function jcsStringify(value: unknown): string { if (!Number.isFinite(value)) throw new TypeError("jcsStringify: non-finite numbers are not representable in JCS"); return JSON.stringify(value); } - if (typeof value === "string") return JSON.stringify(value); + if (typeof value === "string") return stringifyJcsString(value); if (Array.isArray(value)) { + assertDenseJsonArray(value); return `[${value.map(jcsStringify).join(",")}]`; } if (typeof value === "object") { + const prototype = Object.getPrototypeOf(value); + if (prototype !== Object.prototype && prototype !== null) { + throw new TypeError("jcsStringify: only plain JSON objects are representable in JCS"); + } const obj = value as Record<string, unknown>; const keys = Object.keys(obj).sort(); - return `{${keys.map((k) => `${JSON.stringify(k)}:${jcsStringify(obj[k])}`).join(",")}}`; + return `{${keys.map((key) => `${stringifyJcsString(key)}:${jcsStringify(obj[key])}`).join(",")}}`; } throw new TypeError(`jcsStringify: unsupported value type ${typeof value}`); } diff --git a/src/lab/conformance/negative-controls.ts b/src/lab/conformance/negative-controls.ts index 66ab70f4e7..3391e9669a 100644 --- a/src/lab/conformance/negative-controls.ts +++ b/src/lab/conformance/negative-controls.ts @@ -50,11 +50,15 @@ export const NEGATIVE_CONTROL_FIXTURES: Array<{ }, { id: "negative.tool-result-order", - defect: "invalid tool-result ordering", + defect: "invalid tool-result correlation after chat-history repair", mutate: (c) => ({ ...c, id: "negative.tool-result-order", - assertions: [{ id: "result", operator: "tool_result_correlates", selector: "/upstream/requests", expected: { call: "/client/response/toolCalls/0/id", result: "/upstream/requests/1/json/messages/1/tool_call_id" }, required: true }], + // Chat history hardening closes the unmatched call with a synthetic result, then + // re-emits the orphan result behind a synthetic assistant call. Inspect the actual + // supplied result at the end of that repaired pair rather than the synthetic close, + // otherwise the negative control accidentally validates the repair and passes. + assertions: [{ id: "result", operator: "tool_result_correlates", selector: "/upstream/requests", expected: { call: "/client/response/toolCalls/0/id", result: "/upstream/requests/1/json/messages/3/tool_call_id" }, required: true }], fixture: { ...c.fixture, bytesUtf8: JSON.stringify({ diff --git a/src/lab/conformance/runner.ts b/src/lab/conformance/runner.ts index 864f887266..cc8bdcc9e8 100644 --- a/src/lab/conformance/runner.ts +++ b/src/lab/conformance/runner.ts @@ -1,7 +1,7 @@ import { discoverScenarios, loadCaseAuthority } from "./manifest"; import { runScenario } from "./executor"; -import { buildNegativeControls } from "./negative-controls"; -import type { ScenarioRunResult } from "./types"; +import { baseCaseForNegativeControl, buildNegativeControls } from "./negative-controls"; +import type { CaseRecord, ScenarioRunResult } from "./types"; import { CL01_SUITES } from "./types"; export interface ConformanceRunSummary { @@ -19,6 +19,8 @@ export interface NegativeControlRunSummary extends ConformanceRunSummary { rejected: number; } +type ScenarioRunner = (caseRecord: CaseRecord) => Promise<ScenarioRunResult>; + export async function runConformanceSuite( suites: readonly string[] = CL01_SUITES, ): Promise<ConformanceRunSummary> { @@ -33,15 +35,24 @@ export async function runConformanceSuite( return { total: results.length, passed, failed: results.length - passed, results }; } -export async function runNegativeControls(): Promise<NegativeControlRunSummary> { +export async function runNegativeControls( + execute: ScenarioRunner = runScenario, +): Promise<NegativeControlRunSummary> { const authority = loadCaseAuthority(); const scenarios = buildNegativeControls(discoverScenarios(authority)); if (scenarios.length === 0) throw new Error("harness_failure: no negative controls discovered"); const results: ScenarioRunResult[] = []; for (const scenario of scenarios) { - results.push(await runScenario(scenario)); + const baseCase = baseCaseForNegativeControl(scenario.id, authority.cases); + const executionScenario = baseCase ? { ...scenario, id: baseCase.id } : scenario; + const result = await execute(executionScenario); + results.push({ ...result, scenarioId: scenario.id }); } - const rejected = results.filter((r) => !r.passed).length; + const rejected = results.filter((r) => ( + !r.passed + && r.classification === "protocol_failure" + && r.secondaryCode === "deterministic_assertion" + )).length; return { total: results.length, passed: rejected, diff --git a/src/lab/fabric/observe.ts b/src/lab/fabric/observe.ts index 7e93d13fd3..90c6ed8ce1 100644 --- a/src/lab/fabric/observe.ts +++ b/src/lab/fabric/observe.ts @@ -8,11 +8,11 @@ import { OUTCOMES, } from "../constants"; import { FAILURE_CLASSIFICATIONS } from "../conformance/types"; -import { fixtureDigest, isSha256Hex, jcsStringify } from "../digest"; +import { fixtureDigest, isSha256Hex, jcsStringify, subjectIdForSubject } from "../digest"; import type { ObservationEvent, RouteSubjectV1, TaskSubjectV1 } from "../events/types"; import { LabValidationError } from "../events/errors"; import { assignEventId, validateSubject } from "../events/validate"; -import { appendLabEventIfAbsent } from "../ledger/store"; +import { withLedgerMutation } from "../ledger/store"; import { ensureLabDirs } from "../paths"; import { FABRIC_EVIDENCE_LAYER, @@ -172,6 +172,11 @@ function validateFabricVerifier(raw: Record<string, unknown>): void { /** Validate non-negative usage counters on a producer outcome. */ function validateFabricUsage(raw: Record<string, unknown>): void { + for (const key of Object.keys(raw)) { + if (!(USAGE_KEYS as readonly string[]).includes(key)) { + throw new FabricTaskError(`unknown usage field ${key}`, "malformed_producer_outcome", "harness"); + } + } for (const key of USAGE_KEYS) { assertNonNegativeIntegerField(raw, key); } @@ -179,6 +184,11 @@ function validateFabricUsage(raw: Record<string, unknown>): void { /** Validate non-negative limit fields on a producer outcome. */ function validateFabricLimits(raw: Record<string, unknown>): void { + for (const key of Object.keys(raw)) { + if (!(LIMIT_KEYS as readonly string[]).includes(key)) { + throw new FabricTaskError(`unknown limit field ${key}`, "malformed_producer_outcome", "harness"); + } + } for (const key of LIMIT_KEYS) { assertNonNegativeIntegerField(raw, key); } @@ -262,12 +272,18 @@ export function assertFabricOutcomeV1(raw: unknown): FabricTaskOutcomeV1 { } catch (error) { wrapValidationError(error); } + if (jcsStringify(taskSubjectObj) !== jcsStringify(taskSubject)) { + throw new FabricTaskError("taskSubject contains undeclared fields", "malformed_producer_outcome", "harness"); + } + if (jcsStringify(routeSubjectObj) !== jcsStringify(routeSubject)) { + throw new FabricTaskError("routeSubject contains undeclared fields", "malformed_producer_outcome", "harness"); + } if (!routeSubjectsMatch(routeSubject, taskSubject.routeSubject)) { throw new FabricTaskError("contradictory route subjects", "layer_subject_mismatch", "harness"); } - assertStringField(obj, "taskClassId"); - assertStringField(obj, "taskClassVersion"); + const taskClassId = assertStringField(obj, "taskClassId"); + const taskClassVersion = assertStringField(obj, "taskClassVersion"); const subjectId = assertStringField(obj, "subjectId"); if (!isSha256Hex(subjectId)) { throw new FabricTaskError("malformed producer outcome: subjectId", "malformed_producer_outcome", "harness"); @@ -277,16 +293,36 @@ export function assertFabricOutcomeV1(raw: unknown): FabricTaskOutcomeV1 { if (!isSha256Hex(taskFixtureDigest) || !isSha256Hex(verifierManifestDigest)) { throw new FabricTaskError("malformed producer outcome: digest field", "malformed_producer_outcome", "harness"); } - assertStringField(obj, "fabricCompatibilityVersion"); + const fabricCompatibilityVersion = assertStringField(obj, "fabricCompatibilityVersion"); const sandboxProfileDigest = assertStringField(obj, "sandboxProfileDigest"); if (!isSha256Hex(sandboxProfileDigest)) { throw new FabricTaskError("malformed producer outcome: sandboxProfileDigest", "malformed_producer_outcome", "harness"); } - assertIntegerField(obj, "startedAt"); - assertIntegerField(obj, "completedAt"); + if (subjectId !== subjectIdForSubject(taskSubject)) { + throw new FabricTaskError("subjectId does not match taskSubject", "malformed_producer_outcome", "harness"); + } + if ( + taskClassId !== taskSubject.taskClassId + || taskClassVersion !== taskSubject.taskClassVersion + || taskFixtureDigest !== taskSubject.taskFixtureDigest + || verifierManifestDigest !== taskSubject.verifierManifestDigest + || fabricCompatibilityVersion !== taskSubject.fabricCompatibilityVersion + || sandboxProfileDigest !== taskSubject.sandboxProfileDigest + ) { + throw new FabricTaskError("task identity fields do not match taskSubject", "malformed_producer_outcome", "harness"); + } + const startedAt = assertNonNegativeIntegerField(obj, "startedAt"); + const completedAt = assertNonNegativeIntegerField(obj, "completedAt"); + if (completedAt < startedAt) { + throw new FabricTaskError("invalid execution timestamps", "malformed_producer_outcome", "harness"); + } validateFabricLimits(assertPlainObject(obj.limits, "limits")); validateFabricUsage(assertPlainObject(obj.usage, "usage")); - validateFabricVerifier(assertPlainObject(obj.verifier, "verifier")); + const verifier = assertPlainObject(obj.verifier, "verifier"); + validateFabricVerifier(verifier); + if (verifier.manifestDigest !== verifierManifestDigest) { + throw new FabricTaskError("verifier manifest digest does not match outcome", "malformed_producer_outcome", "harness"); + } if (!OUTCOMES.includes(obj.outcome as typeof OUTCOMES[number])) { throw new FabricTaskError("malformed producer outcome: outcome", "malformed_producer_outcome", "harness"); } @@ -324,9 +360,6 @@ export function observationFromFabricOutcome( if (routeSubject.subjectKind !== "route") { throw new FabricTaskError("nested route subject required", "layer_subject_mismatch", "harness"); } - if (!Number.isInteger(outcome.startedAt) || !Number.isInteger(outcome.completedAt) || outcome.completedAt < outcome.startedAt) { - throw new FabricTaskError("invalid execution timestamps", "malformed_producer_outcome", "harness"); - } const paths = ensureLabDirs(opts.configDir); const ownsStore = !opts.artifactStore; @@ -430,9 +463,11 @@ function persistFabricOutcome( const ownsStore = !opts.artifactStore; const store = opts.artifactStore ?? createArtifactStore(paths.artifactsDir); try { - const { event } = observationFromFabricOutcome(outcome, { ...opts, artifactStore: store }); - appendLabEventIfAbsent(paths.ledgerPath, event); - return { event, ledgerPath: paths.ledgerPath }; + return withLedgerMutation(paths.ledgerPath, (ledger) => { + const { event } = observationFromFabricOutcome(outcome, { ...opts, artifactStore: store }); + ledger.appendIfAbsent(event); + return { event, ledgerPath: paths.ledgerPath }; + }); } finally { if (ownsStore) store.close(); } diff --git a/src/lab/index.ts b/src/lab/index.ts index 783eb05526..7f7dad7b34 100644 --- a/src/lab/index.ts +++ b/src/lab/index.ts @@ -36,3 +36,19 @@ export * from "./subject/installation-salt"; export { CL03_LIVE_SUITES } from "./conformance/types"; export * from "./query"; export * from "./automation"; +export * from "./public/types"; +export { + exportLocalPublicEvidence, + importCommunityEvidenceFile, + importCommunityEvidenceValue, + listCommunityEvidenceContext, + previewLocalPublicEvidence, + summarizePublicEvidenceVerification, + verifyPublicEvidenceFile, + type LocalPublicExportV1, + type LocalPublicPreviewV1, + type PublicOperatorExclusionReason, + type PublicOperatorExclusionV1, + type PublicVerificationSummaryV1, +} from "./public/operator"; +export { PublicEvidenceValidationError } from "./public/validate"; diff --git a/src/lab/ledger/purge.ts b/src/lab/ledger/purge.ts index dd6b95853f..4ec55709ab 100644 --- a/src/lab/ledger/purge.ts +++ b/src/lab/ledger/purge.ts @@ -18,10 +18,11 @@ import { expandSensitiveArtifactEventTargets, } from "./artifact-refs"; import { buildInvalidationIndex } from "./invalidation"; -import { appendLabEvent, replayLabLedger } from "./store"; +import { withLedgerMutation } from "./store"; import { ensureLabDirs } from "../paths"; import { rebuildLabProjection } from "../projection/rebuild"; import { jcsStringify } from "../digest"; +import { purgeLocalPublicEvidenceCopies } from "../public/purge"; import { closeSync, existsSync, @@ -34,7 +35,7 @@ import { writeSync, } from "node:fs"; import { dirname, join } from "node:path"; - + export class PurgeError extends Error { readonly code: string; readonly completedActions: string[]; @@ -45,7 +46,7 @@ export class PurgeError extends Error { this.completedActions = [...completedActions]; } } - + export interface SensitivePurgeRequest { configDir?: string; targetEventIds?: string[]; @@ -54,7 +55,7 @@ export interface SensitivePurgeRequest { recordedAt?: number; producerVersion?: string; } - + function writeAll(fd: number, bytes: Uint8Array): void { let offset = 0; while (offset < bytes.byteLength) { @@ -63,7 +64,7 @@ function writeAll(fd: number, bytes: Uint8Array): void { offset += n; } } - + function atomicRewriteLedger(ledgerPath: string, events: LabEvent[]): void { const body = events.map((e) => jcsStringify(e)).join("\n") + (events.length ? "\n" : ""); const bytes = new TextEncoder().encode(body); @@ -90,7 +91,7 @@ function atomicRewriteLedger(ledgerPath: string, events: LabEvent[]): void { } throw err; } - + if (process.platform !== "win32") { try { const dirFd = openSync(parent, "r"); @@ -100,8 +101,6 @@ function atomicRewriteLedger(ledgerPath: string, events: LabEvent[]): void { closeSync(dirFd); } } catch (err) { - // The rename is already committed and visible. Report durability failure - // without pretending the ledger action can be rolled back. throw new PurgeError( "ledger_durability_failed", `ledger rewrite committed but directory fsync failed: ${err instanceof Error ? err.message : String(err)}`, @@ -110,7 +109,7 @@ function atomicRewriteLedger(ledgerPath: string, events: LabEvent[]): void { } } } - + function deleteArtifactsFailClosed(dir: TrustedArtifactDir, digests: string[]): void { const errors: string[] = []; for (const digest of digests) { @@ -125,7 +124,7 @@ function deleteArtifactsFailClosed(dir: TrustedArtifactDir, digests: string[]): throw new PurgeError("artifact_delete_failed", errors.join("; ")); } } - + function purgeBoundedDirectory(dirPath: string): void { if (!existsSync(dirPath)) return; const entries = readdirSync(dirPath, { withFileTypes: true }); @@ -141,7 +140,41 @@ function purgeBoundedDirectory(dirPath: string): void { } } } - + +function normalizePurgeError(err: unknown, completed: readonly string[]): PurgeError { + if (err instanceof PurgeError) { + return new PurgeError( + err.code, + err.message, + [...new Set([...completed, ...err.completedActions])], + ); + } + return new PurgeError( + "purge_failed", + err instanceof Error ? err.message : String(err), + [...completed], + ); +} + +function buildPurgeTombstone( + req: SensitivePurgeRequest, + removeIds: ReadonlySet<string>, + targetArtifactDigests: string[], + purgeActions: Array<(typeof PURGE_ACTIONS)[number]>, +): PurgeTombstoneEvent { + return validateLabEvent(assignEventId({ + schemaVersion: LAB_EVENT_SCHEMA_VERSION, + eventKind: "purge_tombstone" as const, + recordedAt: req.recordedAt ?? Date.now(), + producer: LAB_PRODUCER, + producerVersion: req.producerVersion ?? LAB_PRODUCER_VERSION, + targetEventIds: [...removeIds].sort(), + targetArtifactDigests, + reason: "sensitive_evidence" as const, + purgeActions, + })) as PurgeTombstoneEvent; +} + /** * Exceptional sensitive-evidence purge: * physically remove targeted JSONL lines and artifacts, append purge_tombstone, @@ -153,89 +186,125 @@ export function purgeSensitiveEvidence(req: SensitivePurgeRequest): PurgeTombsto const targetArtifactDigests = [...(req.targetArtifactDigests ?? [])].sort(); const purgeActions = [...(req.purgeActions ?? PURGE_ACTIONS)].sort(); const explicitSensitive = new Set(targetArtifactDigests); - - const replay = replayLabLedger(paths.ledgerPath); - const index = buildInvalidationIndex(replay.events); - const removeIds = expandSensitiveArtifactEventTargets( - replay.events, - index, - new Set(targetEventIds), - explicitSensitive, - ); - - const tombstonePayload = { - schemaVersion: LAB_EVENT_SCHEMA_VERSION, - eventKind: "purge_tombstone" as const, - recordedAt: req.recordedAt ?? Date.now(), - producer: LAB_PRODUCER, - producerVersion: req.producerVersion ?? LAB_PRODUCER_VERSION, - targetEventIds: [...removeIds].sort(), - targetArtifactDigests, - reason: "sensitive_evidence" as const, - purgeActions, - }; - const tombstone = validateLabEvent(assignEventId(tombstonePayload)) as PurgeTombstoneEvent; - - const deletionPlan = purgeActions.includes("artifact") - ? artifactDeletionPlan(replay.events, index, removeIds, targetArtifactDigests) - : { deletable: [], retainedExplicit: [] }; - - if (deletionPlan.retainedExplicit.length > 0) { - throw new PurgeError( - "sensitive_bytes_retained", - `explicit sensitive artifacts remain required: ${deletionPlan.retainedExplicit.join(",")}`, - ); - } - - let dir: TrustedArtifactDir | null = null; const completed: string[] = []; + // Cell object: the mutation callback assigns through the property, which keeps + // the post-mutation read at the declared type (a closure-captured let would + // narrow to null and break the combined-error report below). + const deferredExport: { error: PurgeError | null } = { error: null }; + let operationError: PurgeError | null = null; + let tombstone: PurgeTombstoneEvent | null = null; + try { - if (purgeActions.includes("scratch")) { - purgeBoundedDirectory(paths.scratchDir); - completed.push("scratch"); - } - if (purgeActions.includes("export")) { - purgeBoundedDirectory(paths.exportDir); - completed.push("export"); - } - - if (purgeActions.includes("artifact")) { - if (deletionPlan.deletable.length > 0) { - dir = openTrustedArtifactDir(paths.artifactsDir); - deleteArtifactsFailClosed(dir, deletionPlan.deletable); + tombstone = withLedgerMutation(paths.ledgerPath, (ledger) => { + // Replay and plan under the same lock as every append. Otherwise an event + // appended after this snapshot can be lost by the atomic rename or can + // start referencing an artifact after the deletion plan was calculated. + const replay = ledger.replay(); + const index = buildInvalidationIndex(replay.events); + const removeIds = expandSensitiveArtifactEventTargets( + replay.events, + index, + new Set(targetEventIds), + explicitSensitive, + ); + + const deletionPlan = purgeActions.includes("artifact") + ? artifactDeletionPlan(replay.events, index, removeIds, targetArtifactDigests) + : { deletable: [], retainedExplicit: [] }; + + if (deletionPlan.retainedExplicit.length > 0) { + throw new PurgeError( + "sensitive_bytes_retained", + `explicit sensitive artifacts remain required: ${deletionPlan.retainedExplicit.join(",")}`, + ); } - completed.push("artifact"); - } - - if (purgeActions.includes("ledger")) { - const kept: LabEvent[] = []; - for (const event of replay.events) { - if (removeIds.has(event.eventId)) continue; - kept.push(event); + + if (purgeActions.includes("scratch")) { + purgeBoundedDirectory(paths.scratchDir); + completed.push("scratch"); } - kept.push(tombstone); - atomicRewriteLedger(paths.ledgerPath, kept); - } else { - appendLabEvent(paths.ledgerPath, tombstone); - } - completed.push("ledger"); - + if (purgeActions.includes("export")) { + try { + purgeLocalPublicEvidenceCopies(req.configDir); + completed.push("export"); + } catch (err) { + // Export deletion is independent from artifact/ledger/sqlite deletion. Keep + // deleting every other requested sensitive copy, then report this failure. + deferredExport.error = normalizePurgeError(err, completed); + } + } + + let dir: TrustedArtifactDir | null = null; + try { + if (purgeActions.includes("artifact")) { + if (deletionPlan.deletable.length > 0) { + dir = openTrustedArtifactDir(paths.artifactsDir); + deleteArtifactsFailClosed(dir, deletionPlan.deletable); + } + completed.push("artifact"); + } + + // Never persist a tombstone claiming that export completed when the export + // purge failed. Other independent actions remain recordable and continue. + const tombstoneActions = deferredExport.error + ? purgeActions.filter((action) => action !== "export") + : purgeActions; + const hasTombstoneTarget = removeIds.size > 0 + || targetArtifactDigests.length > 0 + || tombstoneActions.includes("scratch") + || tombstoneActions.includes("export"); + + if (tombstoneActions.length > 0 && (hasTombstoneTarget || !deferredExport.error)) { + const mutationTombstone = buildPurgeTombstone(req, removeIds, targetArtifactDigests, tombstoneActions); + if (purgeActions.includes("ledger")) { + const kept: LabEvent[] = []; + for (const event of replay.events) { + if (removeIds.has(event.eventId)) continue; + kept.push(event); + } + kept.push(mutationTombstone); + atomicRewriteLedger(paths.ledgerPath, kept); + completed.push("ledger"); + } else { + ledger.append(mutationTombstone); + } + return mutationTombstone; + } + return null; + } finally { + if (dir) closeTrustedArtifactDir(dir); + } + }); + + // SQLite is disposable and rebuildLabProjection replays the canonical + // ledger again, so it does not need to extend the mutation lock duration. if (purgeActions.includes("sqlite")) { rebuildLabProjection(req.configDir); completed.push("sqlite"); } - - return tombstone; } catch (err) { - if (err instanceof PurgeError) { - throw new PurgeError(err.code, err.message, [...completed, ...err.completedActions]); - } + operationError = normalizePurgeError(err, completed); + } + + const deferredExportError = deferredExport.error; + if (operationError && deferredExportError) { throw new PurgeError( "purge_failed", - err instanceof Error ? err.message : String(err), - completed, + `export purge failed: ${deferredExportError.message}; subsequent purge failure (${operationError.code}): ${operationError.message}`, + [...new Set([...completed, ...deferredExportError.completedActions, ...operationError.completedActions])], ); - } finally { - if (dir) closeTrustedArtifactDir(dir); } + if (operationError) throw operationError; + if (deferredExportError) { + throw new PurgeError( + deferredExportError.code, + deferredExportError.message, + [...new Set([...completed, ...deferredExportError.completedActions])], + ); + } + if (!tombstone) { + throw new PurgeError("purge_failed", "purge completed without a durable tombstone", completed); + } + return tombstone; } + diff --git a/src/lab/ledger/store.ts b/src/lab/ledger/store.ts index 14baf2c943..cd1d37fc4e 100644 --- a/src/lab/ledger/store.ts +++ b/src/lab/ledger/store.ts @@ -25,6 +25,12 @@ export interface LedgerStore { replay(): ReplayResult; } +export interface LedgerMutationContext { + replay(): ReplayResult; + append(event: LabEvent): void; + appendIfAbsent(event: LabEvent): boolean; +} + const LEDGER_LOCK_STALE_MS = 60_000; const LEDGER_LOCK_WAIT_MS = 5_000; @@ -36,10 +42,7 @@ interface LedgerLockMeta { /** Block synchronously for the given duration (ledger lock retry only). */ function sleepSyncMs(ms: number): void { - const end = Date.now() + ms; - while (Date.now() < end) { - /* spin */ - } + Bun.sleepSync(ms); } /** Read pid, createdAt, and token metadata from a ledger lock file, if well-formed. */ @@ -83,8 +86,94 @@ function isLedgerLockStale(lockPath: string): boolean { return false; } } - if (isLockHolderAlive(meta.pid)) return false; - return Date.now() - meta.createdAt > LEDGER_LOCK_STALE_MS; + return !isLockHolderAlive(meta.pid); +} + +/** Write lock ownership metadata to a newly created exclusive lock file. */ +function writeLedgerLockMeta(fd: number, token: string): void { + const metadataBytes = Buffer.from(JSON.stringify({ + pid: process.pid, + createdAt: Date.now(), + token, + }), "utf8"); + let written = 0; + while (written < metadataBytes.byteLength) { + const n = writeSync(fd, metadataBytes, written, metadataBytes.byteLength - written); + if (n <= 0) { + throw new LabValidationError("short_write", "ledger lock metadata write incomplete"); + } + written += n; + } +} + +/** Discard a lock whose exclusive creator failed before publishing ownership metadata. */ +function discardUninitialisedLedgerLock(lockPath: string, lockFd: number): void { + try { + closeSync(lockFd); + } catch { + /* ignore */ + } + try { + unlinkSync(lockPath); + } catch { + /* best-effort */ + } +} + +/** Release a lock file only when the token still matches the path owner. */ +function releaseLedgerLock(lockPath: string, lockFd: number, token: string): void { + try { + closeSync(lockFd); + } catch { + /* ignore */ + } + try { + const meta = readLedgerLockMeta(lockPath); + if (meta?.token === token) unlinkSync(lockPath); + } catch { + /* best-effort */ + } +} + +/** + * Recover one stale lock while holding a separate recovery mutex. + * + * The recovery mutex prevents two waiters from both observing the same stale + * owner and then unlinking each other's replacement lock. If a process dies + * while holding the recovery mutex, acquisition fails closed instead of + * guessing ownership of that mutex. + */ +function recoverStaleLedgerLock(lockPath: string): boolean { + const recoveryPath = `${lockPath}.recovery`; + const token = randomBytes(16).toString("hex"); + let recoveryFd: number; + try { + recoveryFd = openSync( + recoveryPath, + fsConstants.O_CREAT | fsConstants.O_EXCL | fsConstants.O_WRONLY, + 0o600, + ); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "EEXIST") return false; + throw error; + } + + try { + writeLedgerLockMeta(recoveryFd, token); + } catch (error) { + discardUninitialisedLedgerLock(recoveryPath, recoveryFd); + throw error; + } + + try { + // Re-check after taking the recovery mutex. Another waiter may already + // have recovered the old lock and installed a live replacement. + if (!existsSync(lockPath) || !isLedgerLockStale(lockPath)) return false; + unlinkSync(lockPath); + return true; + } finally { + releaseLedgerLock(recoveryPath, recoveryFd, token); + } } /** Create a ledger lock file exclusively, recovering stale locks when needed. */ @@ -97,30 +186,19 @@ function tryAcquireLedgerLock(lockPath: string, deadline: number): { fd: number; } catch (error) { if (existsSync(lockPath) && isLedgerLockStale(lockPath)) { try { - unlinkSync(lockPath); - } catch (unlinkError) { - if (Date.now() >= deadline) throw unlinkError; - sleepSyncMs(10); + if (recoverStaleLedgerLock(lockPath)) continue; + } catch (recoveryError) { + if (Date.now() >= deadline) throw recoveryError; } - continue; } if (Date.now() >= deadline) throw error; sleepSyncMs(10); continue; } try { - const metadataBytes = Buffer.from(JSON.stringify({ pid: process.pid, createdAt: Date.now(), token }), "utf8"); - const written = writeSync(fd, metadataBytes); - if (written !== metadataBytes.byteLength) { - throw new LabValidationError("short_write", "ledger lock metadata write incomplete"); - } + writeLedgerLockMeta(fd, token); } catch (error) { - closeSync(fd); - try { - unlinkSync(lockPath); - } catch { - /* best-effort */ - } + discardUninitialisedLedgerLock(lockPath, fd); throw error; } return { fd, token }; @@ -128,21 +206,6 @@ function tryAcquireLedgerLock(lockPath: string, deadline: number): { fd: number; throw new Error("ledger lock acquisition timed out"); } -/** Release a ledger lock only when the token still matches the lock file. */ -function releaseLedgerLock(lockPath: string, lockFd: number, token: string): void { - try { - closeSync(lockFd); - } catch { - /* ignore */ - } - try { - const meta = readLedgerLockMeta(lockPath); - if (meta?.token === token) unlinkSync(lockPath); - } catch { - /* best-effort */ - } -} - /** Run a ledger mutation while holding the compatibility ledger lock file. */ function withLedgerLock<T>(ledgerPath: string, fn: () => T): T { const lockPath = `${ledgerPath}.lock`; @@ -156,11 +219,10 @@ function withLedgerLock<T>(ledgerPath: string, fn: () => T): T { } } -/** Durable append of one validated event as a single JSONL line + fsync. */ -export function appendLabEvent(ledgerPath: string, event: LabEvent): void { - const validated = validateLabEvent(event); +/** Durable append of one already-validated event as a single JSONL line + fsync. */ +function appendValidatedLabEvent(ledgerPath: string, event: LabEvent): void { mkdirSync(dirname(ledgerPath), { recursive: true, mode: 0o700 }); - const line = `${jcsStringify(validated)}\n`; + const line = `${jcsStringify(event)}\n`; const bytes = new TextEncoder().encode(line); const fd = openSync(ledgerPath, "a", 0o600); try { @@ -178,26 +240,78 @@ export function appendLabEvent(ledgerPath: string, event: LabEvent): void { } } +function isThenable(value: unknown): value is PromiseLike<unknown> { + return ( + (typeof value === "object" && value !== null) || typeof value === "function" + ) && typeof (value as { then?: unknown }).then === "function"; +} + /** - * Append only when eventId is absent. Uses an exclusive lock file plus a - * process-local event-id index refreshed under the lock. + * Serialize a ledger read-modify-write transaction with all ordinary appends. + * The callback is intentionally synchronous. Mutation methods become invalid + * as soon as the callback returns, so an accidental async continuation cannot + * write after the lock has been released. */ -export function appendLabEventIfAbsent(ledgerPath: string, event: LabEvent): boolean { - const validated = validateLabEvent(event); +export function withLedgerMutation<T>( + ledgerPath: string, + fn: (mutation: LedgerMutationContext) => T, +): T { return withLedgerLock(ledgerPath, () => { - // Refresh from disk under the lock so concurrent writers are visible. - const fresh = new Set<string>(); - if (existsSync(ledgerPath)) { - for (const row of replayLabLedger(ledgerPath).events) { - fresh.add(row.eventId); + let active = true; + const requireActive = () => { + if (!active) { + throw new LabValidationError( + "inactive_ledger_mutation", + "ledger mutation context used after its lock was released", + ); } + }; + const replay = () => { + requireActive(); + return replayLabLedger(ledgerPath); + }; + const append = (event: LabEvent) => { + requireActive(); + appendValidatedLabEvent(ledgerPath, validateLabEvent(event)); + }; + const appendIfAbsent = (event: LabEvent): boolean => { + requireActive(); + const validated = validateLabEvent(event); + if (replay().events.some((row) => row.eventId === validated.eventId)) return false; + appendValidatedLabEvent(ledgerPath, validated); + return true; + }; + + try { + const result = fn({ replay, append, appendIfAbsent }); + if (isThenable(result)) { + throw new LabValidationError( + "async_ledger_mutation", + "ledger mutation callback must be synchronous", + ); + } + return result; + } finally { + active = false; } - if (fresh.has(validated.eventId)) return false; - appendLabEvent(ledgerPath, validated); - return true; }); } +/** Durable append of one validated event as a single JSONL line + fsync. */ +export function appendLabEvent(ledgerPath: string, event: LabEvent): void { + withLedgerMutation(ledgerPath, (mutation) => { + mutation.append(event); + }); +} + +/** + * Append only when eventId is absent. Uses the same mutation lock as every + * other ledger writer so the presence check and append are one transaction. + */ +export function appendLabEventIfAbsent(ledgerPath: string, event: LabEvent): boolean { + return withLedgerMutation(ledgerPath, (mutation) => mutation.appendIfAbsent(event)); +} + function processLine( line: string, lineNumber: number, @@ -415,4 +529,4 @@ export function openLedgerStore(configDir?: string): LedgerStore { export function defaultLedgerPath(configDir?: string): string { return labLedgerPath(configDir); -} +} \ No newline at end of file diff --git a/src/lab/observe/from-conformance.ts b/src/lab/observe/from-conformance.ts index ba353f3050..486391e041 100644 --- a/src/lab/observe/from-conformance.ts +++ b/src/lab/observe/from-conformance.ts @@ -17,7 +17,7 @@ import { } from "../digest"; import type { ObservationEvent } from "../events/types"; import { assignEventId } from "../events/validate"; -import { appendLabEvent } from "../ledger/store"; +import { withLedgerMutation } from "../ledger/store"; import { ensureLabDirs } from "../paths"; import type { CaseAuthority, @@ -285,12 +285,14 @@ export function persistConformanceResult( const ownsStore = !opts.artifactStore; const store = opts.artifactStore ?? createArtifactStore(paths.artifactsDir); try { - const { event } = observationFromConformanceResult(result, caseRecord, authority, { - ...opts, - artifactStore: store, + return withLedgerMutation(paths.ledgerPath, (ledger) => { + const { event } = observationFromConformanceResult(result, caseRecord, authority, { + ...opts, + artifactStore: store, + }); + ledger.append(event); + return { event, ledgerPath: paths.ledgerPath }; }); - appendLabEvent(paths.ledgerPath, event); - return { event, ledgerPath: paths.ledgerPath }; } finally { if (ownsStore) store.close(); } diff --git a/src/lab/observe/from-live.ts b/src/lab/observe/from-live.ts index 857874ed49..08145d9277 100644 --- a/src/lab/observe/from-live.ts +++ b/src/lab/observe/from-live.ts @@ -5,7 +5,7 @@ import { LAB_EVENT_SCHEMA_VERSION, LAB_PRODUCER, LAB_PRODUCER_VERSION, OBSERVATI import { fixtureDigest, scenarioManifestDigest, subjectIdForSubject, suiteManifestDigest } from "../digest"; import type { FailureRecordV1, ObservationEvent } from "../events/types"; import { assignEventId } from "../events/validate"; -import { appendLabEvent } from "../ledger/store"; +import { withLedgerMutation } from "../ledger/store"; import { ensureLabDirs } from "../paths"; import type { CaseAuthority, CaseRecord } from "../conformance/types"; import { trustedLiveResultRetryable } from "../live/executor"; @@ -106,6 +106,12 @@ export function observationFromLiveResult(result: LiveScenarioRunResult, caseRec export function persistLiveResult(result: LiveScenarioRunResult, caseRecord: CaseRecord, authority: CaseAuthority, opts: PersistLiveOptions = {}): PersistedLiveObservation { const paths = ensureLabDirs(opts.configDir); const ownsStore = !opts.artifactStore; const store = opts.artifactStore ?? createArtifactStore(paths.artifactsDir); - try { const { event } = observationFromLiveResult(result, caseRecord, authority, { ...opts, artifactStore: store }); appendLabEvent(paths.ledgerPath, event); return { event, ledgerPath: paths.ledgerPath }; } + try { + return withLedgerMutation(paths.ledgerPath, (ledger) => { + const { event } = observationFromLiveResult(result, caseRecord, authority, { ...opts, artifactStore: store }); + ledger.append(event); + return { event, ledgerPath: paths.ledgerPath }; + }); + } finally { if (ownsStore) store.close(); } } diff --git a/src/lab/paths.ts b/src/lab/paths.ts index f202f4121c..fa39148a9c 100644 --- a/src/lab/paths.ts +++ b/src/lab/paths.ts @@ -81,10 +81,25 @@ export function labScratchDir(configDir = getConfigDir()): string { return join(labRoot(configDir), "scratch"); } +/** Shared Lab export directory. Public evidence bundles intentionally live here too. */ export function labExportDir(configDir = getConfigDir()): string { return join(labRoot(configDir), "export"); } +export function labCommunityDir(configDir = getConfigDir()): string { + return join(labRoot(configDir), "community"); +} + +export function labPublicOriginDir(configDir = getConfigDir()): string { + return join(labRoot(configDir), "public-origin-v1"); +} + +export const LAB_PUBLIC_PUBLISHER_KEY_FILE = "publisher-ed25519.pem"; + +export function labPublicPublisherKeyPath(configDir = getConfigDir()): string { + return join(labRoot(configDir), LAB_PUBLIC_PUBLISHER_KEY_FILE); +} + /** Opaque per-installation salt for local fingerprinting (never exported as evidence). */ export function labInstallationSaltPath(configDir = getConfigDir()): string { return join(labRoot(configDir), "installation-salt.bin"); @@ -110,15 +125,21 @@ export function ensureLabDirs(configDir = getConfigDir()): { artifactsDir: string; scratchDir: string; exportDir: string; + communityDir: string; + publicOriginDir: string; } { const root = labRoot(configDir); const artifactsDir = labArtifactsDir(configDir); const scratchDir = labScratchDir(configDir); const exportDir = labExportDir(configDir); + const communityDir = labCommunityDir(configDir); + const publicOriginDir = labPublicOriginDir(configDir); ensureRestrictedDir(root, root); ensureRestrictedDir(artifactsDir, root); ensureRestrictedDir(scratchDir, root); ensureRestrictedDir(exportDir, root); + ensureRestrictedDir(communityDir, root); + ensureRestrictedDir(publicOriginDir, root); return { root, ledgerPath: labLedgerPath(configDir), @@ -126,5 +147,7 @@ export function ensureLabDirs(configDir = getConfigDir()): { artifactsDir, scratchDir, exportDir, + communityDir, + publicOriginDir, }; } diff --git a/src/lab/public/bundle.ts b/src/lab/public/bundle.ts new file mode 100644 index 0000000000..9bedd41120 --- /dev/null +++ b/src/lab/public/bundle.ts @@ -0,0 +1,217 @@ +import { jcsStringify } from "../digest"; +import { publicEvidenceId } from "./ids"; +import { + PUBLIC_EVIDENCE_BUNDLE_SCHEMA_VERSION, + PUBLIC_EXPORT_POLICY_VERSION, + type PublicArtifactV1, + type PublicEvidenceBundleUnsignedV1, + type PublicEvidenceRecordV1, + type PublicPublisherV1, +} from "./types"; +import { PublicEvidenceValidationError, validatePublicEvidenceRecord } from "./validate"; + +export const MAX_PUBLIC_BUNDLE_BYTES = 2 * 1024 * 1024; +export const MAX_PUBLIC_BUNDLE_RECORDS = 256; +export const MAX_PUBLIC_BUNDLE_ARTIFACTS = 16; +export const MAX_PUBLIC_ARTIFACT_BYTES = 256 * 1024; +export const MAX_PUBLIC_ARTIFACT_BYTES_TOTAL = 1024 * 1024; + +export interface PublicEvidenceContentInput { + records: PublicEvidenceRecordV1[]; + artifacts: PublicArtifactV1[]; + createdDayUtc: string; +} + +export interface BuildPublicEvidenceBundleInput extends PublicEvidenceContentInput { + publisher: PublicPublisherV1; +} + +/** Deterministic, locale-independent code-unit ordering for canonical identity. */ +function compareCanonicalId(a: string, b: string): number { + if (a < b) return -1; + if (a > b) return 1; + return 0; +} + +function utcDay(value: string): string { + if (!/^\d{4}-\d{2}-\d{2}$/.test(value)) { + throw new PublicEvidenceValidationError("invalid_day", "createdDayUtc must be YYYY-MM-DD"); + } + const parsed = new Date(`${value}T00:00:00.000Z`); + if (Number.isNaN(parsed.getTime()) || parsed.toISOString().slice(0, 10) !== value) { + throw new PublicEvidenceValidationError("invalid_day", "createdDayUtc must be a real UTC day"); + } + return value; +} + +function validatePublisher(publisher: PublicPublisherV1): PublicPublisherV1 { + const raw = publisher as unknown as Record<string, unknown>; + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + throw new PublicEvidenceValidationError("invalid_publisher", "publisher must be an object"); + } + const keys = Object.keys(raw); + if (keys.some((key) => !["algorithm", "keyId", "publicKey"].includes(key))) { + throw new PublicEvidenceValidationError("unknown_field", "publisher contains unknown fields"); + } + if (publisher.algorithm !== "ed25519") { + throw new PublicEvidenceValidationError("unsupported_algorithm", "publisher must use ed25519"); + } + if (!/^[0-9a-f]{64}$/.test(publisher.keyId)) { + throw new PublicEvidenceValidationError("invalid_publisher", "publisher.keyId must be sha256 hex"); + } + if (typeof publisher.publicKey !== "string" || publisher.publicKey.length === 0 || publisher.publicKey.length > 1024) { + throw new PublicEvidenceValidationError("invalid_publisher", "publisher.publicKey is invalid"); + } + const publicKeyBytes = Buffer.from(publisher.publicKey, "base64"); + if (publicKeyBytes.byteLength === 0 || publicKeyBytes.toString("base64") !== publisher.publicKey) { + throw new PublicEvidenceValidationError("invalid_publisher", "publisher.publicKey must use canonical base64"); + } + const expectedKeyId = publicEvidenceId("publisher_key", { + algorithm: publisher.algorithm, + publicKey: publisher.publicKey, + }); + if (publisher.keyId !== expectedKeyId) { + throw new PublicEvidenceValidationError("publisher_key_id_mismatch", "publisher.keyId does not match public key"); + } + return { algorithm: "ed25519", keyId: publisher.keyId, publicKey: publisher.publicKey }; +} + +function validateArtifacts(artifacts: PublicArtifactV1[]): PublicArtifactV1[] { + if (!Array.isArray(artifacts) || artifacts.length > MAX_PUBLIC_BUNDLE_ARTIFACTS) { + throw new PublicEvidenceValidationError("array_too_large", `artifacts exceeds ${MAX_PUBLIC_BUNDLE_ARTIFACTS}`); + } + let aggregate = 0; + const ids = new Set<string>(); + return artifacts.map((artifact, index) => { + const raw = artifact as unknown as Record<string, unknown>; + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + throw new PublicEvidenceValidationError("invalid_artifact", `artifacts[${index}] must be an object`); + } + if (Object.keys(raw).some((key) => !["artifactId", "artifactClass", "mediaType", "byteCount", "contentBase64"].includes(key))) { + throw new PublicEvidenceValidationError("unknown_field", `artifacts[${index}] contains unknown fields`); + } + if (!/^[0-9a-f]{64}$/.test(artifact.artifactId)) { + throw new PublicEvidenceValidationError("invalid_artifact", `artifacts[${index}].artifactId is invalid`); + } + if (typeof artifact.artifactClass !== "string" || !/^[A-Za-z0-9][A-Za-z0-9._:+-]{0,255}$/.test(artifact.artifactClass)) { + throw new PublicEvidenceValidationError("invalid_artifact", `artifacts[${index}].artifactClass is invalid`); + } + if (typeof artifact.mediaType !== "string" || artifact.mediaType.length === 0 || artifact.mediaType.length > 256) { + throw new PublicEvidenceValidationError("invalid_artifact", `artifacts[${index}].mediaType is invalid`); + } + if (!Number.isInteger(artifact.byteCount) || artifact.byteCount < 0 || artifact.byteCount > MAX_PUBLIC_ARTIFACT_BYTES) { + throw new PublicEvidenceValidationError("artifact_too_large", `artifacts[${index}].byteCount is invalid`); + } + let bytes: Buffer; + try { + bytes = Buffer.from(artifact.contentBase64, "base64"); + } catch { + throw new PublicEvidenceValidationError("invalid_artifact", `artifacts[${index}].contentBase64 is invalid`); + } + if (bytes.byteLength !== artifact.byteCount || bytes.toString("base64") !== artifact.contentBase64) { + throw new PublicEvidenceValidationError("invalid_artifact", `artifacts[${index}] byte count or base64 is non-canonical`); + } + const expectedArtifactId = publicEvidenceId("artifact", { + artifactClass: artifact.artifactClass, + mediaType: artifact.mediaType, + byteCount: artifact.byteCount, + contentBase64: artifact.contentBase64, + }); + if (artifact.artifactId !== expectedArtifactId) { + throw new PublicEvidenceValidationError("artifact_id_mismatch", `artifacts[${index}].artifactId mismatch`); + } + aggregate += artifact.byteCount; + if (aggregate > MAX_PUBLIC_ARTIFACT_BYTES_TOTAL) { + throw new PublicEvidenceValidationError("artifact_aggregate_too_large", "artifact aggregate exceeds 1 MiB"); + } + if (ids.has(artifact.artifactId)) { + throw new PublicEvidenceValidationError("duplicate_id", "artifacts contains duplicate ids"); + } + ids.add(artifact.artifactId); + return { ...artifact }; + }); +} + +/** Validate all publisher-independent bundle content before any signing-key state is touched. */ +export function normalizePublicEvidenceContent(input: PublicEvidenceContentInput): PublicEvidenceContentInput { + if (!Array.isArray(input.records) || input.records.length > MAX_PUBLIC_BUNDLE_RECORDS) { + throw new PublicEvidenceValidationError("array_too_large", `records exceeds ${MAX_PUBLIC_BUNDLE_RECORDS}`); + } + const records = input.records + .map(validatePublicEvidenceRecord) + .sort((a, b) => compareCanonicalId(a.recordId, b.recordId)); + if (new Set(records.map((record) => record.recordId)).size !== records.length) { + throw new PublicEvidenceValidationError("duplicate_id", "records contains duplicate ids"); + } + const artifacts = validateArtifacts(input.artifacts) + .sort((a, b) => compareCanonicalId(a.artifactId, b.artifactId)); + const artifactIds = new Set(artifacts.map((artifact) => artifact.artifactId)); + for (const record of records) { + for (const artifactId of record.artifactRefs ?? []) { + if (!artifactIds.has(artifactId)) { + throw new PublicEvidenceValidationError("artifact_ref_missing", `record ${record.recordId} references a missing public artifact`); + } + } + } + return { records, artifacts, createdDayUtc: utcDay(input.createdDayUtc) }; +} + +export function canonicalPublicEvidenceContent( + input: PublicEvidenceContentInput, +): { canonical: boolean; normalized: PublicEvidenceContentInput } { + const normalized = normalizePublicEvidenceContent(input); + const canonical = input.records.length === normalized.records.length + && input.artifacts.length === normalized.artifacts.length + && input.records.every((record, index) => record.recordId === normalized.records[index]!.recordId) + && input.artifacts.every((artifact, index) => artifact.artifactId === normalized.artifacts[index]!.artifactId); + return { canonical, normalized }; +} + +export function hasCanonicalPublicEvidenceOrder(input: PublicEvidenceContentInput): boolean { + return canonicalPublicEvidenceContent(input).canonical; +} + +function buildFromNormalizedContent( + normalized: PublicEvidenceContentInput, + publisherInput: PublicPublisherV1, +): PublicEvidenceBundleUnsignedV1 { + const publisher = validatePublisher(publisherInput); + const content = { + schemaVersion: PUBLIC_EVIDENCE_BUNDLE_SCHEMA_VERSION, + exportPolicyVersion: PUBLIC_EXPORT_POLICY_VERSION, + createdDayUtc: normalized.createdDayUtc, + publisher, + records: normalized.records, + artifacts: normalized.artifacts, + }; + const bundleId = publicEvidenceId("bundle", content); + const bundleDigest = publicEvidenceId("bundle_digest", { ...content, bundleId }); + const bundle: PublicEvidenceBundleUnsignedV1 = { ...content, bundleId, bundleDigest }; + if (new TextEncoder().encode(jcsStringify(bundle)).byteLength > MAX_PUBLIC_BUNDLE_BYTES) { + throw new PublicEvidenceValidationError("bundle_too_large", "public bundle exceeds 2 MiB"); + } + return bundle; +} + +export function buildPublicEvidenceBundle(input: BuildPublicEvidenceBundleInput): PublicEvidenceBundleUnsignedV1 { + return buildFromNormalizedContent(normalizePublicEvidenceContent(input), input.publisher); +} + +export function expectedPublicBundleIdentityFromNormalized( + normalized: PublicEvidenceContentInput, + publisher: PublicPublisherV1, +): { bundleId: string; bundleDigest: string } { + const rebuilt = buildFromNormalizedContent(normalized, publisher); + return { bundleId: rebuilt.bundleId, bundleDigest: rebuilt.bundleDigest }; +} + +export function expectedPublicBundleIdentity(bundle: PublicEvidenceBundleUnsignedV1): { bundleId: string; bundleDigest: string } { + return expectedPublicBundleIdentityFromNormalized( + normalizePublicEvidenceContent({ + records: bundle.records, + artifacts: bundle.artifacts, + createdDayUtc: bundle.createdDayUtc, + }), + bundle.publisher, + ); +} diff --git a/src/lab/public/community-authority.ts b/src/lab/public/community-authority.ts new file mode 100644 index 0000000000..2072d29608 --- /dev/null +++ b/src/lab/public/community-authority.ts @@ -0,0 +1,175 @@ +import { loadCaseAuthority } from "../conformance/manifest"; +import { + FABRIC_COMPATIBILITY_VERSION, + FABRIC_SCENARIO_ID, + FABRIC_SCENARIO_VERSION, + FABRIC_SUITE_ID, + FABRIC_SUITE_VERSION, + FABRIC_TASK_CLASS_ID, + FABRIC_TASK_CLASS_VERSION, +} from "../fabric/constants"; +import { loadFabricCaseAuthority } from "../fabric/manifest"; +import { verifierManifestDigest } from "../fabric/subject"; +import { findPublicRouteRegistryEntry } from "./registry"; +import type { PublicEvidenceBundleV1, PublicEvidenceRecordV1, PublicRouteSubjectV1 } from "./types"; +import { PublicEvidenceValidationError } from "./validate"; + +type ProtocolCaseAuthority = ReturnType<typeof loadCaseAuthority>; + +interface ProtocolAuthoritySnapshot { + scenarioVersion: string; + suiteVersion: string; + sourceCommit: string; + load: () => ProtocolCaseAuthority; +} + +// Public records are historical evidence. Never replace an authority entry when a +// protocol version advances: retain the old loader and append a new snapshot. +const PROTOCOL_AUTHORITY_SNAPSHOTS: readonly ProtocolAuthoritySnapshot[] = Object.freeze([ + Object.freeze({ + scenarioVersion: "1.0.0", + suiteVersion: "1.0.0", + sourceCommit: "3ad5bb6bd3f76f6879d84b78ea39edd3e01ec296", + load: loadCaseAuthority, + }), +]); + +const cachedCaseAuthorities = new Map<string, ProtocolCaseAuthority>(); +let cachedFabricCaseAuthority: ReturnType<typeof loadFabricCaseAuthority> | null = null; +let cachedVerifierManifestDigest: string | null = null; + +function protocolAuthorityKey(snapshot: ProtocolAuthoritySnapshot): string { + return `${snapshot.suiteVersion}\0${snapshot.scenarioVersion}`; +} + +function caseAuthorityFor(record: PublicEvidenceRecordV1): ProtocolCaseAuthority { + const snapshot = PROTOCOL_AUTHORITY_SNAPSHOTS.find((candidate) => + candidate.scenarioVersion === record.scenarioVersion + && candidate.suiteVersion === record.suiteVersion + ); + if (!snapshot) { + throw new PublicEvidenceValidationError( + "public_authority", + "scenario/suite authority version is not retained", + ); + } + + const key = protocolAuthorityKey(snapshot); + const cached = cachedCaseAuthorities.get(key); + if (cached) return cached; + + const authority = snapshot.load(); + if ( + String(authority.manifestDefaults.version) !== snapshot.scenarioVersion + || String(authority.manifestDefaults.suiteVersion) !== snapshot.suiteVersion + || authority.sourceCommit !== snapshot.sourceCommit + ) { + throw new Error("public protocol authority snapshot drift"); + } + cachedCaseAuthorities.set(key, authority); + return authority; +} + +function fabricCaseAuthority(): ReturnType<typeof loadFabricCaseAuthority> { + cachedFabricCaseAuthority ??= loadFabricCaseAuthority(); + return cachedFabricCaseAuthority; +} + +function reviewedVerifierManifestDigest(): string { + cachedVerifierManifestDigest ??= verifierManifestDigest(); + return cachedVerifierManifestDigest; +} + +function validateRouteAuthority(subject: PublicRouteSubjectV1): void { + const entry = findPublicRouteRegistryEntry(subject.providerId, subject.modelId); + if (!entry || !entry.adapterFamilies.includes(subject.adapterFamily)) { + throw new PublicEvidenceValidationError("public_authority", "public route is not in reviewed registry authority"); + } +} + +function validateAssertionAuthority( + record: PublicEvidenceRecordV1, + assertions: readonly { id: string; required: boolean }[], +): void { + const allowed = new Map(assertions.map((assertion) => [assertion.id, assertion.required] as const)); + if (allowed.size !== assertions.length) { + throw new PublicEvidenceValidationError("public_authority", "reviewed scenario assertion authority contains duplicates"); + } + if (record.assertions.length !== allowed.size) { + throw new PublicEvidenceValidationError( + "public_authority", + "public assertion set does not exactly match reviewed scenario authority", + ); + } + const seen = new Set<string>(); + for (const assertion of record.assertions) { + if (seen.has(assertion.id)) { + throw new PublicEvidenceValidationError("public_authority", "public assertion set contains duplicate assertion ids"); + } + seen.add(assertion.id); + if (!allowed.has(assertion.id) || allowed.get(assertion.id) !== assertion.required) { + throw new PublicEvidenceValidationError( + "public_authority", + "public assertion id/required flag is not in reviewed scenario authority", + ); + } + } + for (const assertionId of allowed.keys()) { + if (!seen.has(assertionId)) { + throw new PublicEvidenceValidationError("public_authority", "public assertion set is missing reviewed scenario authority"); + } + } +} + +function validateTaskAuthority(record: PublicEvidenceRecordV1): void { + const fabricAuthority = fabricCaseAuthority(); + const caseRecord = fabricAuthority.cases.find((candidate) => candidate.id === FABRIC_SCENARIO_ID); + if ( + !caseRecord + || record.suiteId !== FABRIC_SUITE_ID + || record.suiteVersion !== FABRIC_SUITE_VERSION + || record.scenarioId !== FABRIC_SCENARIO_ID + || record.scenarioVersion !== FABRIC_SCENARIO_VERSION + || record.subject.subjectKind !== "task" + || record.subject.taskClassId !== FABRIC_TASK_CLASS_ID + || record.subject.taskClassVersion !== FABRIC_TASK_CLASS_VERSION + || record.subject.taskFixtureDigest !== caseRecord.fixture.digest + || record.subject.verifierManifestDigest !== reviewedVerifierManifestDigest() + || record.subject.fabricCompatibilityVersion !== FABRIC_COMPATIBILITY_VERSION + ) { + throw new PublicEvidenceValidationError("public_authority", "task scenario/verifier authority mismatch"); + } + validateAssertionAuthority(record, caseRecord.assertions); + validateRouteAuthority(record.subject.route); +} + +function validateScenarioAuthority(record: PublicEvidenceRecordV1): void { + if (record.evidenceLayer === "task_effectiveness") { + validateTaskAuthority(record); + return; + } + + const authority = caseAuthorityFor(record); + const caseRecord = authority.cases.find((candidate) => candidate.id === record.scenarioId); + if (!caseRecord || caseRecord.suite !== record.suiteId) { + throw new PublicEvidenceValidationError("public_authority", "scenario/suite authority mismatch"); + } + validateAssertionAuthority(record, caseRecord.assertions); + + if (record.evidenceLayer === "live_route_compatibility") { + if (record.subject.subjectKind !== "route") { + throw new PublicEvidenceValidationError("public_authority", "live route subject mismatch"); + } + validateRouteAuthority(record.subject); + } +} + +/** Repository-owned authority gate used by both local signing and community imports. */ +export function validatePublicEvidenceAuthorities(records: readonly PublicEvidenceRecordV1[]): void { + for (const record of records) validateScenarioAuthority(record); +} + +export function validateCommunityEvidenceAuthorities(bundle: PublicEvidenceBundleV1): PublicEvidenceBundleV1 { + validatePublicEvidenceAuthorities(bundle.records); + return bundle; +} diff --git a/src/lab/public/community-files.ts b/src/lab/public/community-files.ts new file mode 100644 index 0000000000..38d78f405e --- /dev/null +++ b/src/lab/public/community-files.ts @@ -0,0 +1,29 @@ +const COMMUNITY_ID_FRAGMENT = "[0-9a-f]{64}"; +const COMMUNITY_BUNDLE_FILE_RE = new RegExp( + `^bundle-(${COMMUNITY_ID_FRAGMENT})-(${COMMUNITY_ID_FRAGMENT})\\.json$`, +); +const COMMUNITY_REVOCATION_FILE_RE = new RegExp( + `^revocation-(${COMMUNITY_ID_FRAGMENT})\\.json$`, +); + +export interface CommunityBundleFileIdentity { + publisherKeyId: string; + bundleId: string; +} + +export function communityBundleFileName(publisherKeyId: string, bundleId: string): string { + return `bundle-${publisherKeyId}-${bundleId}.json`; +} + +export function communityRevocationFileName(revocationId: string): string { + return `revocation-${revocationId}.json`; +} + +export function parseCommunityBundleFileName(name: string): CommunityBundleFileIdentity | null { + const match = COMMUNITY_BUNDLE_FILE_RE.exec(name); + return match ? { publisherKeyId: match[1]!, bundleId: match[2]! } : null; +} + +export function isCommunityRevocationFileName(name: string): boolean { + return COMMUNITY_REVOCATION_FILE_RE.test(name); +} diff --git a/src/lab/public/community.ts b/src/lab/public/community.ts new file mode 100644 index 0000000000..4a6d0d6b12 --- /dev/null +++ b/src/lab/public/community.ts @@ -0,0 +1,479 @@ +import { lstatSync, readdirSync, unlinkSync } from "node:fs"; +import { join } from "node:path"; +import { jcsStringify } from "../digest"; +import { ensureLabDirs, labCommunityDir } from "../paths"; +import { validateCommunityEvidenceAuthorities } from "./community-authority"; +import { communityBundleFileName } from "./community-files"; +import { privateRegularFileSize, readPrivateRegularFile } from "./file-safety"; +import { withPublicEvidenceMutationLock } from "./mutation-lock"; +import { recordLocalPublicOrigin } from "./origin"; +import { + cleanupStalePrivateFileStages, + cleanupStalePrivateFileStagesInDir, + isPrivateFileStageName, + publishPrivateFileExclusive, +} from "./private-file"; +import { validatePublicEvidencePrivacy } from "./privacy"; +import { verifyPublicEvidenceRevocation } from "./revocation"; +import { loadExistingPublicPublisher, verifyPublicEvidenceBundle } from "./signature"; +import { parseStrictPublicJson } from "./strict-json"; +import type { + CommunityEvidenceSummaryV1, + PublicEvidenceBundleV1, + PublicEvidenceRevocationV1, +} from "./types"; +import { PublicEvidenceValidationError } from "./validate"; + +const MAX_IMPORT_BYTES = 2 * 1024 * 1024; +const MAX_CACHE_FILES = 512; +const MAX_CACHE_BYTES = 64 * 1024 * 1024; +const MAX_DEPTH = 8; +const MAX_OBJECT_KEYS = 64; +const MAX_ARRAY_ELEMENTS = 512; +const MAX_GENERIC_STRING_BYTES = 384 * 1024; +const COMMUNITY_MUTATION_LOCK_NAME = ".mutation-lock"; +const COMMUNITY_BUNDLE_FILE_RE = /^bundle-([0-9a-f]{64})-([0-9a-f]{64})\.json$/; +const COMMUNITY_REVOCATION_FILE_RE = /^revocation-([0-9a-f]{64})\.json$/; + +const COMMUNITY_FILE_OPTIONS = { + maxBytes: MAX_IMPORT_BYTES, + errorCode: "community_unsafe_target", + errorMessage: "community object is not a bounded private regular file", + sizeErrorCode: "community_size", + sizeErrorMessage: "community file exceeds bound", +} as const; + +type CommunitySummaryCache = { + directory: string; + fingerprint: string; + evidence: CommunityEvidenceSummaryV1[]; +}; + +let communitySummaryCache: CommunitySummaryCache | null = null; + +function assertId(value: string): string { + if (!/^[0-9a-f]{64}$/.test(value)) { + throw new PublicEvidenceValidationError("community_id", "community object id invalid"); + } + return value; +} + +function scanStructure(value: unknown, depth = 0): void { + if (depth > MAX_DEPTH) { + throw new PublicEvidenceValidationError("community_depth", "community JSON nesting depth exceeded"); + } + if (typeof value === "string") { + if (new TextEncoder().encode(value).byteLength > MAX_GENERIC_STRING_BYTES || value.includes("\0")) { + throw new PublicEvidenceValidationError("community_string", "community string invalid or oversized"); + } + return; + } + if (Array.isArray(value)) { + if (value.length > MAX_ARRAY_ELEMENTS) { + throw new PublicEvidenceValidationError("community_array", "community array bound exceeded"); + } + for (const item of value) scanStructure(item, depth + 1); + return; + } + if (value && typeof value === "object") { + const keys = Object.keys(value); + if (keys.length > MAX_OBJECT_KEYS) { + throw new PublicEvidenceValidationError("community_object", "community object key bound exceeded"); + } + for (const key of keys) { + if (new TextEncoder().encode(key).byteLength > 4096) { + throw new PublicEvidenceValidationError("community_key", "community key oversized"); + } + scanStructure((value as Record<string, unknown>)[key], depth + 1); + } + } +} + +function boundedInput(raw: unknown): unknown { + let bytes: Buffer; + if (raw instanceof Uint8Array) { + bytes = Buffer.from(raw); + } else if (typeof raw === "string") { + bytes = Buffer.from(raw, "utf8"); + } else { + scanStructure(raw); + bytes = Buffer.from(jcsStringify(raw), "utf8"); + } + if (bytes.byteLength > MAX_IMPORT_BYTES) { + throw new PublicEvidenceValidationError("community_size", "community import exceeds 2 MiB"); + } + const parsed = parseStrictPublicJson(bytes, "community import"); + scanStructure(parsed); + return parsed; +} + +function assertCommunityArtifactAuthority(bundle: PublicEvidenceBundleV1): void { + if (bundle.artifacts.length !== 0) { + throw new PublicEvidenceValidationError( + "public_artifact_authority_required", + "community artifact bytes require reviewed public_export policy authority", + ); + } +} + +function verifiedBundle(raw: unknown): PublicEvidenceBundleV1 { + const result = verifyPublicEvidenceBundle(raw as PublicEvidenceBundleV1); + if (result.status !== "cryptographically_valid") { + throw new PublicEvidenceValidationError(result.status, "community bundle verification failed"); + } + const bundle = validateCommunityEvidenceAuthorities(raw as PublicEvidenceBundleV1); + assertCommunityArtifactAuthority(bundle); + validatePublicEvidencePrivacy(bundle); + return bundle; +} + +function bundleObjectPath(publisherKeyId: string, bundleId: string, configDir?: string): string { + return join( + labCommunityDir(configDir), + communityBundleFileName(assertId(publisherKeyId), assertId(bundleId)), + ); +} + +function revocationObjectPath(revocationId: string, configDir?: string): string { + return join(labCommunityDir(configDir), `revocation-${assertId(revocationId)}.json`); +} + +function readBounded(path: string): Buffer { + cleanupStalePrivateFileStages(path); + return readPrivateRegularFile(path, COMMUNITY_FILE_OPTIONS); +} + +function cacheUsage(configDir?: string): { names: string[]; bytes: number } { + ensureLabDirs(configDir); + const dir = labCommunityDir(configDir); + cleanupStalePrivateFileStagesInDir(dir); + const names = readdirSync(dir) + .filter((name) => name !== COMMUNITY_MUTATION_LOCK_NAME && !isPrivateFileStageName(name)) + .sort(); + if (names.length > MAX_CACHE_FILES) { + throw new PublicEvidenceValidationError("community_cache_bound", "community cache file bound exceeded"); + } + let bytes = 0; + for (const name of names) { + bytes += privateRegularFileSize(join(dir, name), COMMUNITY_FILE_OPTIONS); + if (bytes > MAX_CACHE_BYTES) { + throw new PublicEvidenceValidationError("community_cache_bound", "community cache byte bound exceeded"); + } + } + return { names, bytes }; +} + +function assertCacheCanAdd(byteCount: number, configDir?: string): void { + const usage = cacheUsage(configDir); + if (usage.names.length >= MAX_CACHE_FILES || usage.bytes + byteCount > MAX_CACHE_BYTES) { + throw new PublicEvidenceValidationError("community_cache_bound", "community cache capacity exceeded"); + } +} + +function persistAtLocked( + path: string, + kind: "bundle" | "revocation", + value: unknown, + configDir?: string, + onCommit?: () => void, +): { path: string; created: boolean } { + const bytes = Buffer.from(jcsStringify(value), "utf8"); + if (bytes.byteLength > MAX_IMPORT_BYTES) { + throw new PublicEvidenceValidationError("community_size", "community object exceeds bound"); + } + + let created = false; + try { + try { + const existing = readBounded(path); + if (!existing.equals(bytes)) { + throw new PublicEvidenceValidationError("community_conflict", `${kind} identity already exists with different bytes`); + } + } catch (error) { + if (error instanceof PublicEvidenceValidationError) throw error; + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + assertCacheCanAdd(bytes.byteLength, configDir); + const published = publishPrivateFileExclusive(path, bytes); + if (!published.created) { + const raced = readBounded(path); + if (!raced.equals(bytes)) { + throw new PublicEvidenceValidationError("community_conflict", `${kind} identity already exists with different bytes`); + } + } else { + created = true; + cacheUsage(configDir); + } + } + + onCommit?.(); + if (created) communitySummaryCache = null; + return { path, created }; + } catch (error) { + if (created) { + try { unlinkSync(path); } catch { /* preserve commit error */ } + communitySummaryCache = null; + } + throw error; + } +} + +function persistAt( + path: string, + kind: "bundle" | "revocation", + value: unknown, + configDir?: string, + onCommit?: () => void, +): { path: string; created: boolean } { + return withPublicEvidenceMutationLock( + configDir, + () => persistAtLocked(path, kind, value, configDir, onCommit), + ); +} + +function readJson(path: string): unknown { + const parsed = parseStrictPublicJson(readBounded(path), "stored community object"); + scanStructure(parsed); + return parsed; +} + +function files(configDir?: string): string[] { + return cacheUsage(configDir).names; +} + +function readVerifiedBundleAt(path: string): PublicEvidenceBundleV1 { + return verifiedBundle(readJson(path)); +} + +function bundleFromName(name: string, configDir?: string): PublicEvidenceBundleV1 | null { + const match = COMMUNITY_BUNDLE_FILE_RE.exec(name); + if (!match) return null; + const publisherKeyId = match[1]!; + const bundleId = match[2]!; + const bundle = readVerifiedBundleAt(bundleObjectPath(publisherKeyId, bundleId, configDir)); + if (bundle.bundleId !== bundleId || bundle.publisher.keyId !== publisherKeyId) { + throw new PublicEvidenceValidationError("community_identity_mismatch", "stored community bundle does not match filename identity"); + } + return bundle; +} + +function bundlesFromNames(names: readonly string[], configDir?: string): PublicEvidenceBundleV1[] { + const bundles: PublicEvidenceBundleV1[] = []; + for (const name of names) { + const bundle = bundleFromName(name, configDir); + if (bundle) bundles.push(bundle); + } + return bundles; +} + +function restoreOwnPublisherOrigin(bundle: PublicEvidenceBundleV1, configDir?: string): void { + const local = loadExistingPublicPublisher(configDir); + if (!local) return; + if ( + local.publisher.algorithm !== bundle.publisher.algorithm + || local.publisher.keyId !== bundle.publisher.keyId + || local.publisher.publicKey !== bundle.publisher.publicKey + ) return; + recordLocalPublicOrigin({ publisherKeyId: bundle.publisher.keyId, bundleId: bundle.bundleId }, configDir); +} + +export function importCommunityEvidenceBundle( + raw: unknown, + configDir?: string, +): { created: boolean; status: "cryptographically_valid"; bundleId: string; publisherKeyId: string; path: string } { + const bundle = verifiedBundle(boundedInput(raw)); + ensureLabDirs(configDir); + const stored = persistAt( + bundleObjectPath(bundle.publisher.keyId, bundle.bundleId, configDir), + "bundle", + bundle, + configDir, + () => restoreOwnPublisherOrigin(bundle, configDir), + ); + return { ...stored, status: "cryptographically_valid", bundleId: bundle.bundleId, publisherKeyId: bundle.publisher.keyId }; +} + +function readCommunityEvidenceBundleForPublisherLocked( + bundleId: string, + publisherKeyId: string, + configDir?: string, +): PublicEvidenceBundleV1 { + const bundle = readVerifiedBundleAt(bundleObjectPath(publisherKeyId, bundleId, configDir)); + if (bundle.bundleId !== bundleId || bundle.publisher.keyId !== publisherKeyId) { + throw new PublicEvidenceValidationError("community_identity_mismatch", "stored community bundle does not match filename identity"); + } + return bundle; +} + +export function readCommunityEvidenceBundleForPublisher( + bundleId: string, + publisherKeyId: string, + configDir?: string, +): PublicEvidenceBundleV1 { + return withPublicEvidenceMutationLock( + configDir, + () => readCommunityEvidenceBundleForPublisherLocked(bundleId, publisherKeyId, configDir), + ); +} + +type RevocationMetadata = { + publisher?: { keyId?: unknown }; + targets?: Array<{ kind?: unknown; id?: unknown }>; +}; + +function resolveTargetBundle( + revocation: unknown, + bundles: readonly PublicEvidenceBundleV1[], +): PublicEvidenceBundleV1 { + if (!revocation || typeof revocation !== "object") { + throw new PublicEvidenceValidationError("revocation_target", "revocation target metadata unavailable"); + } + const raw = revocation as RevocationMetadata; + if (!Array.isArray(raw.targets) || typeof raw.publisher?.keyId !== "string") { + throw new PublicEvidenceValidationError("revocation_target", "revocation targets or publisher unavailable"); + } + const publisherKeyId = assertId(raw.publisher.keyId); + const publisherBundles = bundles + .filter((bundle) => bundle.publisher.keyId === publisherKeyId) + .sort((a, b) => a.bundleId.localeCompare(b.bundleId)); + const bundleTargets = raw.targets.filter((target) => target.kind === "bundle" && typeof target.id === "string"); + if (bundleTargets.length > 0) { + const targetIds = new Set(bundleTargets.map((target) => target.id)); + if (targetIds.size !== 1) { + throw new PublicEvidenceValidationError("revocation_target", "revocation bundle targets are ambiguous"); + } + const id = [...targetIds][0]!; + const candidate = publisherBundles.find((bundle) => bundle.bundleId === id); + if (!candidate) throw new PublicEvidenceValidationError("revocation_target", "revocation target bundle not found"); + return candidate; + } + const fullyMatching = publisherBundles.filter((bundle) => raw.targets!.every((target) => + target.kind === "record" && typeof target.id === "string" + && bundle.records.some((record) => record.recordId === target.id), + )); + if (fullyMatching.length === 0) { + throw new PublicEvidenceValidationError( + "revocation_target", + "revocation targets do not resolve to a verified bundle for the same publisher", + ); + } + return fullyMatching[0]!; +} + +function findTargetBundleLocked(revocation: unknown, configDir?: string): PublicEvidenceBundleV1 { + const names = files(configDir); + const raw = revocation as RevocationMetadata; + const publisherKeyId = typeof raw?.publisher?.keyId === "string" ? assertId(raw.publisher.keyId) : null; + const directBundleIds = Array.isArray(raw?.targets) + ? [...new Set(raw.targets.filter((target) => target.kind === "bundle" && typeof target.id === "string").map((target) => target.id as string))] + : []; + if (publisherKeyId && directBundleIds.length === 1) { + try { + return readCommunityEvidenceBundleForPublisherLocked( + assertId(directBundleIds[0]!), + publisherKeyId, + configDir, + ); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") { + throw new PublicEvidenceValidationError( + "revocation_target", + "revocation target bundle not found", + ); + } + throw error; + } + } + return resolveTargetBundle(revocation, bundlesFromNames(names, configDir)); +} + +export function importCommunityEvidenceRevocation( + raw: unknown, + configDir?: string, +): { created: boolean; status: "cryptographically_valid"; revocationId: string; path: string } { + const parsed = boundedInput(raw); + ensureLabDirs(configDir); + return withPublicEvidenceMutationLock(configDir, () => { + const targetBundle = findTargetBundleLocked(parsed, configDir); + const verified = verifyPublicEvidenceRevocation(parsed, targetBundle); + if (verified.status !== "cryptographically_valid") { + throw new PublicEvidenceValidationError(verified.status, verified.detail ?? "community revocation verification failed"); + } + const stored = persistAtLocked( + revocationObjectPath(verified.revocation.revocationId, configDir), + "revocation", + verified.revocation, + configDir, + ); + return { ...stored, status: "cryptographically_valid", revocationId: verified.revocation.revocationId }; + }); +} + +function communityFingerprint(names: readonly string[], configDir?: string): string { + const dir = labCommunityDir(configDir); + return names.map((name) => { + const stat = lstatSync(join(dir, name)); + return [name, stat.dev, stat.ino, stat.mode, stat.nlink, stat.size, stat.mtimeMs, stat.ctimeMs].join(":"); + }).join("\n"); +} + +function copySummaries(evidence: readonly CommunityEvidenceSummaryV1[]): CommunityEvidenceSummaryV1[] { + return evidence.map((row) => ({ ...row })); +} + +function listCommunityEvidenceLocked(configDir?: string): CommunityEvidenceSummaryV1[] { + const names = files(configDir); + const directory = labCommunityDir(configDir); + const fingerprint = communityFingerprint(names, configDir); + if (communitySummaryCache?.directory === directory && communitySummaryCache.fingerprint === fingerprint) { + return copySummaries(communitySummaryCache.evidence); + } + + const bundles = bundlesFromNames(names, configDir); + const revocations: PublicEvidenceRevocationV1[] = []; + + for (const name of names) { + if (!COMMUNITY_REVOCATION_FILE_RE.test(name)) continue; + const raw = readJson(join(directory, name)); + let targetBundle: PublicEvidenceBundleV1; + try { + targetBundle = resolveTargetBundle(raw, bundles); + } catch (error) { + if (error instanceof PublicEvidenceValidationError) continue; + throw error; + } + const verified = verifyPublicEvidenceRevocation(raw, targetBundle); + if (verified.status === "cryptographically_valid") revocations.push(verified.revocation); + } + + const evidence = bundles.map((bundle) => { + const revoked = new Set<string>(); + const bundleRecordIds = new Set(bundle.records.map((record) => record.recordId)); + for (const revocation of revocations) { + if (revocation.publisher.keyId !== bundle.publisher.keyId + || revocation.publisher.publicKey !== bundle.publisher.publicKey) { + continue; + } + if (revocation.targets.some((target) => target.kind === "bundle" && target.id === bundle.bundleId)) { + for (const record of bundle.records) revoked.add(record.recordId); + } + for (const target of revocation.targets) { + if (target.kind === "record" && bundleRecordIds.has(target.id)) revoked.add(target.id); + } + } + return { + trustClass: "community_untrusted_v1" as const, + status: "cryptographically_valid" as const, + bundleId: bundle.bundleId, + publisherKeyId: bundle.publisher.keyId, + activeRecordCount: bundle.records.filter((record) => !revoked.has(record.recordId)).length, + revokedRecordCount: bundle.records.filter((record) => revoked.has(record.recordId)).length, + }; + }).sort((a, b) => a.bundleId.localeCompare(b.bundleId) || a.publisherKeyId.localeCompare(b.publisherKeyId)); + + communitySummaryCache = { directory, fingerprint, evidence: copySummaries(evidence) }; + return copySummaries(evidence); +} + +export function listCommunityEvidence(configDir?: string): CommunityEvidenceSummaryV1[] { + return withPublicEvidenceMutationLock(configDir, () => listCommunityEvidenceLocked(configDir)); +} diff --git a/src/lab/public/file-safety.ts b/src/lab/public/file-safety.ts new file mode 100644 index 0000000000..78134929cf --- /dev/null +++ b/src/lab/public/file-safety.ts @@ -0,0 +1,155 @@ +import { + closeSync, + constants as fsConstants, + fstatSync, + fsyncSync, + lstatSync, + openSync, + readFileSync, + readdirSync, + unlinkSync, +} from "node:fs"; +import { basename, dirname, join } from "node:path"; +import { isPrivateFileStageName } from "./private-file"; +import { PublicEvidenceValidationError } from "./validate"; + +const O_NOFOLLOW = (fsConstants as { O_NOFOLLOW?: number }).O_NOFOLLOW ?? 0; + +export interface PrivateRegularFileReadOptions { + maxBytes: number; + errorCode: string; + errorMessage: string; + sizeErrorCode?: string; + sizeErrorMessage?: string; + requireMode600?: boolean; +} + +function sizeError(options: PrivateRegularFileReadOptions): PublicEvidenceValidationError { + return new PublicEvidenceValidationError( + options.sizeErrorCode ?? options.errorCode, + options.sizeErrorMessage ?? options.errorMessage, + ); +} + +/** + * Heal only the publication-specific hard link left behind when the final name was linked + * but the parent-directory durability check failed. The stage must be target-scoped and + * inode-identical to the final file; unrelated hard links remain and are rejected below. + */ +function recoverPublishedPrivateFileStage(path: string): void { + if (process.platform === "win32") return; + const finalStats = lstatSync(path); + if (finalStats.isSymbolicLink() || !finalStats.isFile() || finalStats.nlink <= 1) return; + + const dir = dirname(path); + const prefix = `.${basename(path)}.`; + const candidates: string[] = []; + for (const name of readdirSync(dir)) { + if (!name.startsWith(prefix) || !isPrivateFileStageName(name)) continue; + const stagePath = join(dir, name); + try { + const stageStats = lstatSync(stagePath); + if ( + stageStats.isFile() + && !stageStats.isSymbolicLink() + && stageStats.dev === finalStats.dev + && stageStats.ino === finalStats.ino + ) { + candidates.push(stagePath); + } + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + } + } + if (candidates.length === 0) return; + + // The final directory entry already exists. Make that entry durable before deleting + // the recovery witness. A real fsync failure propagates and the strict read stays closed. + let dirFd: number | null = null; + try { + dirFd = openSync(dir, fsConstants.O_RDONLY); + fsyncSync(dirFd); + } finally { + if (dirFd !== null) closeSync(dirFd); + } + + for (const stagePath of candidates) { + try { + const stageStats = lstatSync(stagePath); + if ( + stageStats.isFile() + && !stageStats.isSymbolicLink() + && stageStats.dev === finalStats.dev + && stageStats.ino === finalStats.ino + ) { + unlinkSync(stagePath); + } + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + } + } +} + +function withPrivateRegularFile<T>( + path: string, + options: PrivateRegularFileReadOptions, + consume: (fd: number, size: number) => T, +): T { + let pathStats = lstatSync(path); + if (!pathStats.isSymbolicLink() && pathStats.isFile() && pathStats.nlink > 1) { + recoverPublishedPrivateFileStage(path); + pathStats = lstatSync(path); + } + if (pathStats.isSymbolicLink() || !pathStats.isFile() || pathStats.nlink !== 1) { + throw new PublicEvidenceValidationError(options.errorCode, options.errorMessage); + } + if (pathStats.size > options.maxBytes) throw sizeError(options); + + const fd = openSync(path, fsConstants.O_RDONLY | O_NOFOLLOW); + try { + const stats = fstatSync(fd); + if ( + !stats.isFile() + || stats.nlink !== 1 + || stats.dev !== pathStats.dev + || stats.ino !== pathStats.ino + ) { + throw new PublicEvidenceValidationError(options.errorCode, options.errorMessage); + } + if (stats.size > options.maxBytes) throw sizeError(options); + if (options.requireMode600 && process.platform !== "win32" && (stats.mode & 0o777) !== 0o600) { + throw new PublicEvidenceValidationError(options.errorCode, options.errorMessage); + } + return consume(fd, stats.size); + } finally { + closeSync(fd); + } +} + +/** + * Inspect a file only after proving that the pathname and checked descriptor refer to + * the same private regular file. This keeps quota scans descriptor-bound without + * reading every cached object into memory. + */ +export function privateRegularFileSize( + path: string, + options: PrivateRegularFileReadOptions, +): number { + return withPrivateRegularFile(path, options, (_fd, size) => size); +} + +/** + * Read bytes only after proving that the pathname and the consumed descriptor refer to + * the same private regular file. The lstat/dev+ino comparison keeps the protection on + * platforms where O_NOFOLLOW is unavailable instead of silently following a symlink. + */ +export function readPrivateRegularFile( + path: string, + options: PrivateRegularFileReadOptions, +): Buffer { + return withPrivateRegularFile(path, options, (fd) => { + const bytes = readFileSync(fd); + if (bytes.byteLength > options.maxBytes) throw sizeError(options); + return bytes; + }); +} diff --git a/src/lab/public/ids.ts b/src/lab/public/ids.ts new file mode 100644 index 0000000000..ff78437545 --- /dev/null +++ b/src/lab/public/ids.ts @@ -0,0 +1,26 @@ +import { domainHash, jcsStringify } from "../digest"; + +export type PublicEvidenceIdKind = + | "subject" + | "record" + | "bundle" + | "bundle_digest" + | "artifact" + | "publisher_key" + | "revocation" + | "route_registry"; + +const PUBLIC_EVIDENCE_DOMAIN: Record<PublicEvidenceIdKind, string> = { + subject: "ocx-lab-public:subject:v1", + record: "ocx-lab-public:record:v1", + bundle: "ocx-lab-public:bundle:v1", + bundle_digest: "ocx-lab-public:bundle-digest:v1", + artifact: "ocx-lab-public:artifact:v1", + publisher_key: "ocx-lab-public:publisher-key:v1", + revocation: "ocx-lab-public:revocation:v1", + route_registry: "ocx-lab-public:route-registry:v1", +}; + +export function publicEvidenceId(kind: PublicEvidenceIdKind, payload: unknown): string { + return domainHash(PUBLIC_EVIDENCE_DOMAIN[kind], jcsStringify(payload)); +} diff --git a/src/lab/public/index.ts b/src/lab/public/index.ts new file mode 100644 index 0000000000..e16890d88c --- /dev/null +++ b/src/lab/public/index.ts @@ -0,0 +1,16 @@ +export * from "./types"; +export * from "./ids"; +export * from "./registry"; +export * from "./validate"; +export * from "./privacy"; +export * from "./project"; +export * from "./bundle"; +export * from "./signature"; +export * from "./storage"; +export * from "./community-authority"; +export * from "./revocation"; +export * from "./community"; +export * from "./strict-json"; +export * from "./origin"; +export * from "./operator"; +export * from "./purge"; diff --git a/src/lab/public/mutation-lock.ts b/src/lab/public/mutation-lock.ts new file mode 100644 index 0000000000..8b52012e28 --- /dev/null +++ b/src/lab/public/mutation-lock.ts @@ -0,0 +1,424 @@ +import { randomUUID } from "node:crypto"; +import { + lstatSync, + mkdirSync, + readdirSync, + renameSync, + rmSync, + unlinkSync, + writeFileSync, +} from "node:fs"; +import { dirname, join } from "node:path"; +import { ensureLabDirs, labCommunityDir } from "../paths"; +import { readPrivateRegularFile } from "./file-safety"; +import { PublicEvidenceValidationError } from "./validate"; + +const PUBLIC_EVIDENCE_MUTATION_LOCK_NAME = ".mutation-lock"; +const PUBLIC_EVIDENCE_MUTATION_LOCK_OWNER = "owner.json"; +const PUBLIC_EVIDENCE_MUTATION_LOCK_RECLAIM = ".reclaim.json"; +const PUBLIC_EVIDENCE_MUTATION_LOCK_INCOMPLETE_STALE_MS = 24 * 60 * 60 * 1000; +const PUBLIC_EVIDENCE_MUTATION_LOCK_ABSOLUTE_STALE_MS = 7 * 24 * 60 * 60 * 1000; +const DETACHED_MUTATION_LOCK_RE = /^\.mutation-lock-(?:stale|release)-\d+-[0-9a-f-]{36}$/; +const MUTATION_LOCK_META_FILE_OPTIONS = { + maxBytes: 1024, + errorCode: "community_cache_lock", + errorMessage: "community cache mutation lock metadata is unsafe", + sizeErrorCode: "community_cache_lock", + sizeErrorMessage: "community cache mutation lock metadata exceeds its size bound", + requireMode600: true, +} as const; + +type MutationLockOwner = { + pid: number; + token: string; + createdAt: number; +}; + +type MutationLockReclaim = { + pid: number; + token: string; + createdAt: number; +}; + +type MutationLockDirectoryIdentity = { + dev: number; + ino: number; +}; + +function mutationLockPath(configDir?: string): string { + return join(labCommunityDir(configDir), PUBLIC_EVIDENCE_MUTATION_LOCK_NAME); +} + +function mutationLockOwnerPath(lockPath: string): string { + return join(lockPath, PUBLIC_EVIDENCE_MUTATION_LOCK_OWNER); +} + +function mutationLockReclaimPath(lockPath: string): string { + return join(lockPath, PUBLIC_EVIDENCE_MUTATION_LOCK_RECLAIM); +} + +function pidDefinitelyDead(pid: number): boolean { + try { + process.kill(pid, 0); + return false; + } catch (error) { + return (error as NodeJS.ErrnoException).code === "ESRCH"; + } +} + +function exceedsAbsoluteLockAge(createdAt: number, nowMs: number): boolean { + return nowMs - createdAt > PUBLIC_EVIDENCE_MUTATION_LOCK_ABSOLUTE_STALE_MS; +} + +function readLockMetadata<T extends MutationLockOwner | MutationLockReclaim>( + path: string, +): T | null { + try { + const bytes = readPrivateRegularFile(path, MUTATION_LOCK_META_FILE_OPTIONS); + const raw = JSON.parse(bytes.toString("utf8")) as Partial<T>; + if ( + Number.isSafeInteger(raw.pid) + && Number(raw.pid) > 0 + && typeof raw.token === "string" + && /^[0-9a-f-]{36}$/.test(raw.token) + && Number.isSafeInteger(raw.createdAt) + && Number(raw.createdAt) > 0 + ) { + return { + pid: Number(raw.pid), + token: raw.token, + createdAt: Number(raw.createdAt), + } as T; + } + } catch { + // Incomplete metadata is handled conservatively by the age fallback where + // applicable. Normal acquisition never trusts malformed metadata. + } + return null; +} + +function readMutationLockOwner(lockPath: string): MutationLockOwner | null { + return readLockMetadata<MutationLockOwner>(mutationLockOwnerPath(lockPath)); +} + +function readMutationLockReclaim(lockPath: string): MutationLockReclaim | null { + return readLockMetadata<MutationLockReclaim>(mutationLockReclaimPath(lockPath)); +} + +function assertMutationLockDirectory(lockPath: string) { + const stat = lstatSync(lockPath); + if (!stat.isDirectory() || stat.isSymbolicLink()) { + throw new PublicEvidenceValidationError( + "community_cache_lock", + "community cache mutation lock is not a directory", + ); + } + return stat; +} + +function sameDirectoryIdentity( + left: MutationLockDirectoryIdentity, + right: MutationLockDirectoryIdentity, +): boolean { + return left.dev === right.dev && left.ino === right.ino; +} + +function currentDirectoryIdentity(lockPath: string): MutationLockDirectoryIdentity { + const stat = assertMutationLockDirectory(lockPath); + return { dev: stat.dev, ino: stat.ino }; +} + +function mutationLockIsReclaimable(lockPath: string, nowMs: number): boolean { + const stat = assertMutationLockDirectory(lockPath); + const owner = readMutationLockOwner(lockPath); + if (owner) { + // A live PID is strong evidence only while the recorded ownership generation is + // reasonably recent. The absolute ceiling recovers from PID reuse after a crash. + return pidDefinitelyDead(owner.pid) || exceedsAbsoluteLockAge(owner.createdAt, nowMs); + } + // The only ownerless state is the tiny mkdir-to-owner publication window. Use + // a deliberately long fallback so a crashed acquisition can eventually heal + // without treating an ordinary pause as proof that the owner disappeared. + return nowMs - stat.mtimeMs > PUBLIC_EVIDENCE_MUTATION_LOCK_INCOMPLETE_STALE_MS; +} + +function unlinkIfPresent(path: string): void { + try { + unlinkSync(path); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + } +} + +function cleanupDetachedMutationLocks(lockPath: string): void { + const dir = dirname(lockPath); + for (const name of readdirSync(dir)) { + if (!DETACHED_MUTATION_LOCK_RE.test(name)) continue; + // Detached lock directories are no longer authoritative once a new canonical + // lock has been acquired. Removing only their UUID-scoped names prevents them + // from leaking storage or being mistaken for community cache objects. + rmSync(join(dir, name), { recursive: true, force: true }); + } +} + +function publishMutationLockOwner( + lockPath: string, + owner: MutationLockOwner, + expectedDirectory: MutationLockDirectoryIdentity, +): void { + // Write through a unique temporary pathname first. If an ancient ownerless + // lock is reclaimed while this process was suspended, the inode check prevents + // this acquisition from publishing its owner metadata into a replacement lock. + const tempPath = join(lockPath, `.owner-${owner.token}.tmp`); + try { + writeFileSync(tempPath, JSON.stringify(owner), { + encoding: "utf8", + flag: "wx", + mode: 0o600, + }); + if (!sameDirectoryIdentity(currentDirectoryIdentity(lockPath), expectedDirectory)) { + throw new PublicEvidenceValidationError( + "community_cache_lock", + "community cache mutation lock changed during owner publication", + ); + } + renameSync(tempPath, mutationLockOwnerPath(lockPath)); + if (!sameDirectoryIdentity(currentDirectoryIdentity(lockPath), expectedDirectory)) { + throw new PublicEvidenceValidationError( + "community_cache_lock", + "community cache mutation lock changed after owner publication", + ); + } + const persisted = readMutationLockOwner(lockPath); + if (!persisted || persisted.pid !== owner.pid || persisted.token !== owner.token) { + throw new PublicEvidenceValidationError( + "community_cache_lock", + "community cache mutation lock owner publication was not durable", + ); + } + } finally { + // The rename normally makes this ENOENT. If the lock pathname was replaced, + // the UUID-scoped temporary name can be removed without touching successor state. + unlinkIfPresent(tempPath); + } +} + +function reclaimClaimIsRecoverable(lockPath: string, nowMs: number): boolean { + const claimPath = mutationLockReclaimPath(lockPath); + const claim = readMutationLockReclaim(lockPath); + if (claim) { + return pidDefinitelyDead(claim.pid) || exceedsAbsoluteLockAge(claim.createdAt, nowMs); + } + try { + const stat = lstatSync(claimPath); + return nowMs - stat.mtimeMs > PUBLIC_EVIDENCE_MUTATION_LOCK_INCOMPLETE_STALE_MS; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return true; + throw error; + } +} + +function recoverStaleReclaimClaim(lockPath: string, nowMs: number): boolean { + if (!reclaimClaimIsRecoverable(lockPath, nowMs)) return false; + const claimPath = mutationLockReclaimPath(lockPath); + const quarantinePath = join(lockPath, `.reclaim-stale-${randomUUID()}.json`); + try { + renameSync(claimPath, quarantinePath); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return true; + throw error; + } + unlinkIfPresent(quarantinePath); + return true; +} + +function tryAcquireReclaimClaim( + lockPath: string, + nowMs: number, +): MutationLockReclaim | null { + for (let attempt = 0; attempt < 2; attempt += 1) { + const claim: MutationLockReclaim = { + pid: process.pid, + token: randomUUID(), + createdAt: nowMs, + }; + try { + writeFileSync(mutationLockReclaimPath(lockPath), JSON.stringify(claim), { + encoding: "utf8", + flag: "wx", + mode: 0o600, + }); + return claim; + } catch (error) { + const code = (error as NodeJS.ErrnoException).code; + if (code === "ENOENT") return null; + if (code !== "EEXIST") throw error; + if (!recoverStaleReclaimClaim(lockPath, nowMs)) return null; + } + } + return null; +} + +function reclaimClaimStillOwned(lockPath: string, claim: MutationLockReclaim): boolean { + const current = readMutationLockReclaim(lockPath); + return current?.pid === claim.pid && current.token === claim.token; +} + +function releaseReclaimClaim(lockPath: string, claim: MutationLockReclaim): void { + if (!reclaimClaimStillOwned(lockPath, claim)) return; + unlinkIfPresent(mutationLockReclaimPath(lockPath)); +} + +function tryReclaimMutationLock(lockPath: string, nowMs: number): boolean { + // Capture the exact stale directory before the claim write changes its mtime. + // After claiming, revalidate the inode and owner instead of reusing an age check + // that our own .reclaim.json creation would make appear fresh. + if (!mutationLockIsReclaimable(lockPath, nowMs)) return false; + const staleDirectory = currentDirectoryIdentity(lockPath); + const claim = tryAcquireReclaimClaim(lockPath, nowMs); + if (!claim) return false; + + let moved = false; + try { + if (!reclaimClaimStillOwned(lockPath, claim)) return false; + if (!sameDirectoryIdentity(currentDirectoryIdentity(lockPath), staleDirectory)) return false; + const currentOwner = readMutationLockOwner(lockPath); + if ( + currentOwner + && !pidDefinitelyDead(currentOwner.pid) + && !exceedsAbsoluteLockAge(currentOwner.createdAt, nowMs) + ) return false; + if (!reclaimClaimStillOwned(lockPath, claim)) return false; + + const quarantinePath = join( + dirname(lockPath), + `.mutation-lock-stale-${process.pid}-${randomUUID()}`, + ); + try { + // Rename the exact claimed directory away from the canonical pathname before + // deleting it. A successor can create a new lock immediately afterwards, but + // cleanup is confined to this unique quarantine path and cannot delete it. + renameSync(lockPath, quarantinePath); + moved = true; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return true; + throw error; + } + rmSync(quarantinePath, { recursive: true, force: true }); + return true; + } finally { + if (!moved) releaseReclaimClaim(lockPath, claim); + } +} + +function discardUncommittedMutationLock( + lockPath: string, + expectedDirectory: MutationLockDirectoryIdentity, +): void { + try { + if (!sameDirectoryIdentity(currentDirectoryIdentity(lockPath), expectedDirectory)) return; + const quarantinePath = join( + dirname(lockPath), + `.mutation-lock-release-${process.pid}-${randomUUID()}`, + ); + renameSync(lockPath, quarantinePath); + rmSync(quarantinePath, { recursive: true, force: true }); + } catch { + // Preserve the owner-publication error. An unrecoverable cleanup witness stays + // ownerless and can be reclaimed by the long incomplete-acquisition fallback. + } +} + +function releaseMutationLock( + lockPath: string, + owner: MutationLockOwner, + expectedDirectory: MutationLockDirectoryIdentity, +): void { + let currentDirectory: MutationLockDirectoryIdentity; + try { + currentDirectory = currentDirectoryIdentity(lockPath); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return; + throw error; + } + if (!sameDirectoryIdentity(currentDirectory, expectedDirectory)) return; + const current = readMutationLockOwner(lockPath); + if (!current || current.pid !== owner.pid || current.token !== owner.token) return; + + const quarantinePath = join( + dirname(lockPath), + `.mutation-lock-release-${process.pid}-${randomUUID()}`, + ); + try { + renameSync(lockPath, quarantinePath); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return; + throw error; + } + rmSync(quarantinePath, { recursive: true, force: true }); +} + +/** Serialize public-evidence mutations across CLI/server processes with ownership-safe stale recovery. */ +export function withPublicEvidenceMutationLock<T>( + configDir: string | undefined, + run: () => T, +): T { + ensureLabDirs(configDir); + const lockPath = mutationLockPath(configDir); + let owner: MutationLockOwner | null = null; + let ownedDirectory: MutationLockDirectoryIdentity | null = null; + + while (true) { + try { + mkdirSync(lockPath, { mode: 0o700 }); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "EEXIST") throw error; + try { + if (tryReclaimMutationLock(lockPath, Date.now())) continue; + } catch (reclaimError) { + if ((reclaimError as NodeJS.ErrnoException).code === "ENOENT") continue; + throw reclaimError; + } + throw new PublicEvidenceValidationError("community_cache_busy", "community cache is busy"); + } + + const directory = currentDirectoryIdentity(lockPath); + const candidate: MutationLockOwner = { + pid: process.pid, + token: randomUUID(), + createdAt: Date.now(), + }; + try { + publishMutationLockOwner(lockPath, candidate, directory); + } catch (error) { + discardUncommittedMutationLock(lockPath, directory); + throw error; + } + owner = candidate; + ownedDirectory = directory; + break; + } + + try { + cleanupDetachedMutationLocks(lockPath); + return run(); + } finally { + if (owner && ownedDirectory) releaseMutationLock(lockPath, owner, ownedDirectory); + } +} + +/** Test-only seam for stale-owner policy. This module is not barrel-exported. */ +export function publicEvidenceMutationLockIsReclaimableForTests( + configDir: string | undefined, + nowMs = Date.now(), +): boolean { + return mutationLockIsReclaimable(mutationLockPath(configDir), nowMs); +} + +/** Test-only seam for the exclusive stale-reclaimer claim. */ +export function publicEvidenceTryReclaimMutationLockForTests( + configDir: string | undefined, + nowMs = Date.now(), +): boolean { + return tryReclaimMutationLock(mutationLockPath(configDir), nowMs); +} diff --git a/src/lab/public/operator.ts b/src/lab/public/operator.ts new file mode 100644 index 0000000000..edcf15de8b --- /dev/null +++ b/src/lab/public/operator.ts @@ -0,0 +1,353 @@ +import { replayLabLedger } from "../ledger/store"; +import { labLedgerPath } from "../paths"; +import { queryLabEvents, queryLabVerdicts } from "../query"; +import type { ObservationEvent } from "../events/types"; +import { + importCommunityEvidenceBundle, + listCommunityEvidence, +} from "./community"; +import { readPrivateRegularFile } from "./file-safety"; +import { withPublicEvidenceMutationLock } from "./mutation-lock"; +import { recordLocalPublicOrigin } from "./origin"; +import type { ProjectPublicEvidenceRecordInput } from "./project"; +import { projectPublicEvidenceRecord } from "./project"; +import { + signPublicEvidenceBundle, + verifyPublicEvidenceBundle, +} from "./signature"; +import { storePublicEvidenceBundle } from "./storage"; +import { parseStrictPublicJson } from "./strict-json"; +import { publicUtcDay } from "./time"; +import { PUBLIC_EVIDENCE_BUNDLE_SCHEMA_VERSION, PUBLIC_EXPORT_POLICY_VERSION } from "./types"; +import type { + PublicEvidenceBundleV1, + PublicEvidencePreviewBundleV1, + PublicEvidenceRecordV1, + PublicProjectionNotExportableReason, +} from "./types"; +import { PublicEvidenceValidationError } from "./validate"; + +const MAX_OPERATOR_EVENTS = 256; +const MAX_PUBLIC_FILE_BYTES = 2 * 1024 * 1024; +const EMPTY_PREVIEW_DAY = "1970-01-01"; +const PRIVATE_STORAGE_LOCATOR = "<private>"; + +export interface ProjectPublicEvidenceInput { + records: ProjectPublicEvidenceRecordInput[]; +} + +export function projectPublicEvidence(input: ProjectPublicEvidenceInput): { + bundle: PublicEvidencePreviewBundleV1; + excluded: Array<{ index: number; reason: PublicProjectionNotExportableReason }>; +} { + const records: PublicEvidenceRecordV1[] = []; + const excluded: Array<{ index: number; reason: PublicProjectionNotExportableReason }> = []; + let latestExportableCompletedAt: number | null = null; + + input.records.forEach((recordInput, index) => { + const projected = projectPublicEvidenceRecord(recordInput); + if (projected.status !== "exportable") { + excluded.push({ index, reason: projected.reason }); + return; + } + records.push(projected.record); + latestExportableCompletedAt = Math.max( + latestExportableCompletedAt ?? recordInput.observation.completedAt, + recordInput.observation.completedAt, + ); + }); + records.sort((a, b) => a.recordId < b.recordId ? -1 : a.recordId > b.recordId ? 1 : 0); + return { + bundle: { + schemaVersion: PUBLIC_EVIDENCE_BUNDLE_SCHEMA_VERSION, + exportPolicyVersion: PUBLIC_EXPORT_POLICY_VERSION, + createdDayUtc: latestExportableCompletedAt === null + ? EMPTY_PREVIEW_DAY + : publicUtcDay(latestExportableCompletedAt), + records, + artifacts: [], + }, + excluded, + }; +} + +export type PublicOperatorExclusionReason = + | PublicProjectionNotExportableReason + | "event_not_found" + | "not_observation" + | "event_excluded" + | "no_canonical_verdict"; + +export interface PublicOperatorExclusionV1 { + selectionIndex: number; + reason: PublicOperatorExclusionReason; +} + +export interface LocalPublicPreviewV1 { + bundle: PublicEvidencePreviewBundleV1; + excluded: PublicOperatorExclusionV1[]; +} + +export interface LocalPublicExportV1 { + bundle: PublicEvidenceBundleV1; + stored: { path: typeof PRIVATE_STORAGE_LOCATOR; created: boolean }; + excluded: PublicOperatorExclusionV1[]; +} + +export type PublicVerificationSummaryV1 = + | { status: "cryptographically_valid"; bundleId: string; publisherKeyId: string; locallyVerified: false } + | { status: "schema_rejected" | "digest_invalid" | "signature_invalid"; locallyVerified: false }; + +function assertOperatorEventIds(eventIds: readonly string[]): Array<{ eventId: string; selectionIndex: number }> { + if (eventIds.length === 0 || eventIds.length > MAX_OPERATOR_EVENTS) { + throw new PublicEvidenceValidationError( + "public_selection_limit", + `public evidence selection must contain 1..${MAX_OPERATOR_EVENTS} event ids`, + ); + } + const unique: Array<{ eventId: string; selectionIndex: number }> = []; + const seen = new Set<string>(); + for (const [selectionIndex, eventId] of eventIds.entries()) { + if (!/^[0-9a-f]{64}$/.test(eventId)) { + throw new PublicEvidenceValidationError( + "public_selection_event_id", + "public evidence event ids must be lowercase sha256 hex", + ); + } + if (seen.has(eventId)) continue; + seen.add(eventId); + unique.push({ eventId, selectionIndex }); + } + return unique; +} + +function projectedObservationState( + eventIds: readonly string[], + configDir?: string, +): Map<string, { excluded: boolean }> { + const pending = new Set(eventIds); + const state = new Map<string, { excluded: boolean }>(); + let cursor: string | undefined; + + while (pending.size > 0) { + const page = queryLabEvents({ eventKind: "observation" }, cursor, 200, configDir); + for (const row of page.items) { + if (!pending.has(row.eventId)) continue; + state.set(row.eventId, { excluded: row.excluded }); + pending.delete(row.eventId); + } + if (!page.hasMore || !page.nextCursor) break; + cursor = page.nextCursor; + } + + return state; +} + +function canonicalVerdictsForObservations( + observations: readonly ObservationEvent[], + configDir?: string, +): Map<string, ProjectPublicEvidenceRecordInput["verdict"]> { + const verdictByEventId = new Map<string, ProjectPublicEvidenceRecordInput["verdict"]>(); + const groups = new Map<string, { + subjectId: string; + evidenceLayer: ObservationEvent["evidenceLayer"]; + suiteId: string; + observations: ObservationEvent[]; + }>(); + + for (const observation of observations) { + const key = `${observation.subjectId}\0${observation.evidenceLayer}\0${observation.suiteId}`; + const existing = groups.get(key); + if (existing) existing.observations.push(observation); + else groups.set(key, { + subjectId: observation.subjectId, + evidenceLayer: observation.evidenceLayer, + suiteId: observation.suiteId, + observations: [observation], + }); + } + + for (const group of groups.values()) { + const pending = new Map(group.observations.map((observation) => [observation.eventId, observation] as const)); + let cursor: string | undefined; + while (pending.size > 0) { + const page = queryLabVerdicts({ + subjectId: group.subjectId, + layer: group.evidenceLayer, + suiteId: group.suiteId, + }, cursor, 200, configDir); + for (const row of page.items) { + for (const eventId of row.contributingEventIds) { + const observation = pending.get(eventId); + if (!observation) continue; + if ( + row.subjectId !== observation.subjectId + || row.evidenceLayer !== observation.evidenceLayer + || row.suiteId !== observation.suiteId + || row.suiteVersion !== observation.suiteVersion + ) { + continue; + } + verdictByEventId.set(eventId, row.verdict); + pending.delete(eventId); + } + } + if (!page.hasMore || !page.nextCursor) break; + cursor = page.nextCursor; + } + } + + return verdictByEventId; +} + +export function previewLocalPublicEvidence( + input: { eventIds: readonly string[] }, + configDir?: string, +): LocalPublicPreviewV1 { + const selections = assertOperatorEventIds(input.eventIds); + const replay = replayLabLedger(labLedgerPath(configDir)); + const byId = new Map(replay.events.map((event) => [event.eventId, event] as const)); + const observationSelections: Array<{ observation: ObservationEvent; selectionIndex: number }> = []; + const excluded: PublicOperatorExclusionV1[] = []; + + for (const { eventId, selectionIndex } of selections) { + const event = byId.get(eventId); + if (!event) { + excluded.push({ selectionIndex, reason: "event_not_found" }); + continue; + } + if (event.eventKind !== "observation") { + excluded.push({ selectionIndex, reason: "not_observation" }); + continue; + } + observationSelections.push({ observation: event, selectionIndex }); + } + + if (observationSelections.length === 0) { + throw new PublicEvidenceValidationError("public_selection_empty", "public evidence selection contains no observation events"); + } + + const projectionByEventId = projectedObservationState( + observationSelections.map(({ observation }) => observation.eventId), + configDir, + ); + const candidates: Array<{ observation: ObservationEvent; selectionIndex: number }> = []; + for (const candidate of observationSelections) { + const projectedEvent = projectionByEventId.get(candidate.observation.eventId); + if (!projectedEvent) { + excluded.push({ selectionIndex: candidate.selectionIndex, reason: "event_not_found" }); + continue; + } + if (projectedEvent.excluded) { + excluded.push({ selectionIndex: candidate.selectionIndex, reason: "event_excluded" }); + continue; + } + candidates.push(candidate); + } + + const verdictByEventId = canonicalVerdictsForObservations( + candidates.map((candidate) => candidate.observation), + configDir, + ); + const projectInputs: ProjectPublicEvidenceRecordInput[] = []; + const projectSelectionIndices: number[] = []; + for (const { observation, selectionIndex } of candidates) { + const verdict = verdictByEventId.get(observation.eventId); + if (!verdict) { + excluded.push({ selectionIndex, reason: "no_canonical_verdict" }); + continue; + } + projectInputs.push({ observation, verdict }); + projectSelectionIndices.push(selectionIndex); + } + + const projected = projectPublicEvidence({ records: projectInputs }); + for (const row of projected.excluded) { + excluded.push({ selectionIndex: projectSelectionIndices[row.index]!, reason: row.reason }); + } + excluded.sort((a, b) => a.selectionIndex - b.selectionIndex); + return { bundle: projected.bundle, excluded }; +} + +export function exportLocalPublicEvidence( + input: { eventIds: readonly string[] }, + configDir?: string, +): LocalPublicExportV1 { + const preview = previewLocalPublicEvidence(input, configDir); + if (preview.bundle.records.length === 0) { + throw new PublicEvidenceValidationError("public_export_empty", "selected events produced no exportable public evidence records"); + } + const bundle = signPublicEvidenceBundle({ + records: preview.bundle.records, + artifacts: preview.bundle.artifacts, + createdDayUtc: preview.bundle.createdDayUtc, + configDir, + }); + return withPublicEvidenceMutationLock(configDir, () => { + // Commit purge-owned provenance first. If export publication later fails or the + // process crashes, an orphan marker is conservative and can be reclaimed later; + // the inverse state, a durable export without provenance, is not acceptable. + recordLocalPublicOrigin({ publisherKeyId: bundle.publisher.keyId, bundleId: bundle.bundleId }, configDir); + const stored = storePublicEvidenceBundle(bundle, configDir); + return { + bundle, + stored: { path: PRIVATE_STORAGE_LOCATOR, created: stored.created }, + excluded: preview.excluded, + }; + }); +} + +export function summarizePublicEvidenceVerification(raw: unknown): PublicVerificationSummaryV1 { + const result = verifyPublicEvidenceBundle(raw as PublicEvidenceBundleV1); + if (result.status !== "cryptographically_valid") { + return { status: result.status, locallyVerified: false }; + } + const bundle = raw as PublicEvidenceBundleV1; + return { + status: "cryptographically_valid", + bundleId: bundle.bundleId, + publisherKeyId: bundle.publisher.keyId, + locallyVerified: false, + }; +} + +function readBoundedPublicFile(path: string): Buffer { + return readPrivateRegularFile(path, { + maxBytes: MAX_PUBLIC_FILE_BYTES, + errorCode: "public_file_unsafe", + errorMessage: "public evidence input must be a regular non-symlink file", + sizeErrorCode: "public_file_too_large", + sizeErrorMessage: "public evidence input exceeds 2 MiB", + }); +} + +function parsePublicFile(path: string): unknown { + return parseStrictPublicJson(readBoundedPublicFile(path), "public evidence input", "public_file_json"); +} + +export function verifyPublicEvidenceFile(path: string): PublicVerificationSummaryV1 { + return summarizePublicEvidenceVerification(parsePublicFile(path)); +} + +function finishCommunityImport( + stored: { path: string; created: boolean; bundleId: string; publisherKeyId: string }, +) { + const { path: _privatePath, ...imported } = stored; + return { ...imported, trustClass: "community_untrusted_v1" as const, locallyVerified: false as const }; +} + +export function importCommunityEvidenceFile(path: string, configDir?: string) { + return finishCommunityImport(importCommunityEvidenceBundle(readBoundedPublicFile(path), configDir)); +} + +export function importCommunityEvidenceValue(raw: unknown, configDir?: string) { + return finishCommunityImport(importCommunityEvidenceBundle(raw, configDir)); +} + +export function listCommunityEvidenceContext(configDir?: string) { + return { + evidence: listCommunityEvidence(configDir), + trustClass: "community_untrusted_v1" as const, + locallyVerified: false as const, + }; +} diff --git a/src/lab/public/origin-purge.ts b/src/lab/public/origin-purge.ts new file mode 100644 index 0000000000..57e6919b8f --- /dev/null +++ b/src/lab/public/origin-purge.ts @@ -0,0 +1,79 @@ +import { readdirSync } from "node:fs"; +import { join } from "node:path"; +import { ensureLabDirs, labPublicOriginDir } from "../paths"; +import { readPrivateRegularFile } from "./file-safety"; +import { cleanupStalePrivateFileStagesInDir, isPrivateFileStageName } from "./private-file"; +import { parseStrictPublicJson } from "./strict-json"; + +const MAX_ORIGIN_BYTES = 1024; +const ORIGIN_RE = /^origin-([0-9a-f]{64})-([0-9a-f]{64})\.json$/; + +export interface PurgeOriginIdentity { + publisherKeyId: string; + bundleId: string; +} + +export interface PurgeOriginRecovery { + identities: PurgeOriginIdentity[]; + skipped: number; +} + +/** + * Purge must salvage each provenance marker independently. A corrupt marker is untrusted + * and skipped, but it cannot hide later valid markers that are needed to classify local + * community copies after the export or publisher key is unavailable. The operational + * 1024-marker quota is deliberately not a read cutoff here: recovery must inspect every + * valid-format marker present after a race/crash instead of silently losing provenance. + */ +export function recoverPublicOriginsForPurge(configDir?: string): PurgeOriginRecovery { + ensureLabDirs(configDir); + const dir = labPublicOriginDir(configDir); + cleanupStalePrivateFileStagesInDir(dir); + const names = readdirSync(dir) + .filter((name) => !isPrivateFileStageName(name) && ORIGIN_RE.test(name)) + .sort(); + const identities: PurgeOriginIdentity[] = []; + let skipped = 0; + + for (const name of names) { + const match = ORIGIN_RE.exec(name)!; + const expected = { publisherKeyId: match[1]!, bundleId: match[2]! }; + try { + const raw = parseStrictPublicJson( + readPrivateRegularFile(join(dir, name), { + maxBytes: MAX_ORIGIN_BYTES, + errorCode: "public_origin_unsafe", + errorMessage: "public origin marker is unsafe during purge", + sizeErrorCode: "public_origin_unsafe", + sizeErrorMessage: "public origin marker exceeds its size bound", + requireMode600: true, + }), + "public origin marker during purge", + "public_origin_json", + ); + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + skipped += 1; + continue; + } + const row = raw as Record<string, unknown>; + if ( + Object.keys(row).sort().join(",") !== "bundleId,publisherKeyId,schemaVersion" + || row.schemaVersion !== "public_origin_v1" + || row.publisherKeyId !== expected.publisherKeyId + || row.bundleId !== expected.bundleId + ) { + skipped += 1; + continue; + } + identities.push(expected); + } catch { + skipped += 1; + // Salvage continues with the next marker. + } + } + return { identities, skipped }; +} + +export function listValidPublicOriginsForPurge(configDir?: string): PurgeOriginIdentity[] { + return recoverPublicOriginsForPurge(configDir).identities; +} diff --git a/src/lab/public/origin.ts b/src/lab/public/origin.ts new file mode 100644 index 0000000000..c0d3b9f2da --- /dev/null +++ b/src/lab/public/origin.ts @@ -0,0 +1,203 @@ +import { lstatSync, readdirSync, unlinkSync } from "node:fs"; +import { join } from "node:path"; +import { jcsStringify } from "../digest"; +import { ensureLabDirs, labCommunityDir, labExportDir, labPublicOriginDir } from "../paths"; +import { communityBundleFileName } from "./community-files"; +import { readPrivateRegularFile } from "./file-safety"; +import { + cleanupStalePrivateFileStagesInDir, + isPrivateFileStageName, + publishPrivateFileExclusive, +} from "./private-file"; +import { parseStrictPublicJson } from "./strict-json"; +import { PublicEvidenceValidationError } from "./validate"; + +// The community cache itself is capped at 512 files. Keeping twice that many origin +// markers leaves headroom for in-flight/local exports while allowing unreferenced +// provenance to be reclaimed instead of permanently locking future exports. +const MAX_ORIGINS = 1024; +const MAX_ORIGIN_BYTES = 1024; +const ORIGIN_RE = /^origin-([0-9a-f]{64})-([0-9a-f]{64})\.json$/; + +export interface PublicOriginIdentityV1 { + publisherKeyId: string; + bundleId: string; +} + +function originPath(identity: PublicOriginIdentityV1, configDir?: string): string { + if (!/^[0-9a-f]{64}$/.test(identity.publisherKeyId) || !/^[0-9a-f]{64}$/.test(identity.bundleId)) { + throw new PublicEvidenceValidationError("public_origin_id", "public origin identity is invalid"); + } + return join( + labPublicOriginDir(configDir), + `origin-${identity.publisherKeyId}-${identity.bundleId}.json`, + ); +} + +function originBody(identity: PublicOriginIdentityV1): Buffer { + return Buffer.from(jcsStringify({ + schemaVersion: "public_origin_v1", + publisherKeyId: identity.publisherKeyId, + bundleId: identity.bundleId, + }), "utf8"); +} + +function readOrigin(path: string, expected?: PublicOriginIdentityV1): PublicOriginIdentityV1 { + const bytes = readPrivateRegularFile(path, { + maxBytes: MAX_ORIGIN_BYTES, + errorCode: "public_origin_unsafe", + errorMessage: "public origin marker is not a private regular file with 0600 permissions", + sizeErrorCode: "public_origin_unsafe", + sizeErrorMessage: "public origin marker exceeds its size bound", + requireMode600: true, + }); + const raw = parseStrictPublicJson(bytes, "public origin marker", "public_origin_json"); + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + throw new PublicEvidenceValidationError("public_origin_json", "public origin marker must be an object"); + } + const row = raw as Record<string, unknown>; + if (Object.keys(row).sort().join(",") !== "bundleId,publisherKeyId,schemaVersion" + || row.schemaVersion !== "public_origin_v1" + || typeof row.publisherKeyId !== "string" + || typeof row.bundleId !== "string" + || !/^[0-9a-f]{64}$/.test(row.publisherKeyId) + || !/^[0-9a-f]{64}$/.test(row.bundleId)) { + throw new PublicEvidenceValidationError("public_origin_json", "public origin marker schema is invalid"); + } + const identity = { publisherKeyId: row.publisherKeyId, bundleId: row.bundleId }; + if (expected && (identity.publisherKeyId !== expected.publisherKeyId || identity.bundleId !== expected.bundleId)) { + throw new PublicEvidenceValidationError("public_origin_conflict", "public origin marker identity mismatch"); + } + return identity; +} + +/** Marker names only. Quota accounting, reclaim, and listing must agree on this set. */ +function originNames(dir: string): string[] { + cleanupStalePrivateFileStagesInDir(dir); + return readdirSync(dir) + .filter((name) => !isPrivateFileStageName(name) && ORIGIN_RE.test(name)) + .sort(); +} + +function foreignOriginNames(dir: string): string[] { + return readdirSync(dir) + .filter((name) => !isPrivateFileStageName(name) && !ORIGIN_RE.test(name)) + .sort(); +} + +function pathExistsConservatively(path: string): boolean { + try { + lstatSync(path); + return true; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return false; + return true; + } +} + +function communityBundlePath(identity: PublicOriginIdentityV1, configDir?: string): string { + return join( + labCommunityDir(configDir), + communityBundleFileName(identity.publisherKeyId, identity.bundleId), + ); +} + +function localExportPath(identity: PublicOriginIdentityV1, configDir?: string): string { + return join(labExportDir(configDir), `${identity.bundleId}.json`); +} + +/** + * Origin markers exist to recover local provenance for community copies when the export + * or publisher key is later unavailable. A marker is reclaimable only when neither the + * exact community copy nor its matching local export still exists. + */ +function reclaimUnreferencedOrigins( + dir: string, + preservePath: string, + configDir?: string, +): void { + for (const name of originNames(dir)) { + const match = ORIGIN_RE.exec(name)!; + const path = join(dir, name); + if (path === preservePath) continue; + const identity = { publisherKeyId: match[1]!, bundleId: match[2]! }; + if (pathExistsConservatively(communityBundlePath(identity, configDir))) continue; + if (pathExistsConservatively(localExportPath(identity, configDir))) continue; + try { + unlinkSync(path); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + } + } +} + +export function recordLocalPublicOrigin(identity: PublicOriginIdentityV1, configDir?: string): void { + ensureLabDirs(configDir); + const dir = labPublicOriginDir(configDir); + const path = originPath(identity, configDir); + try { + readOrigin(path, identity); + return; + } catch (error) { + if (error instanceof PublicEvidenceValidationError) throw error; + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + } + + let names = originNames(dir); + if (names.length >= MAX_ORIGINS) { + reclaimUnreferencedOrigins(dir, path, configDir); + names = originNames(dir); + } + if (names.length >= MAX_ORIGINS) { + throw new PublicEvidenceValidationError("public_origin_bound", "public origin marker bound exceeded"); + } + + const bytes = originBody(identity); + const published = publishPrivateFileExclusive(path, bytes); + if (!published.created) { + readOrigin(path, identity); + return; + } + + // Separate CLI processes can both observe one free slot before either publishes. + // Reclaim unreferenced history after publication, then remove only this call's marker + // if the directory still cannot converge inside the hard cap. + if (originNames(dir).length > MAX_ORIGINS) { + reclaimUnreferencedOrigins(dir, path, configDir); + if (originNames(dir).length > MAX_ORIGINS) { + try { unlinkSync(path); } catch { /* preserve the quota failure */ } + throw new PublicEvidenceValidationError("public_origin_bound", "public origin marker bound exceeded"); + } + } +} + +export function listLocalPublicOrigins(configDir?: string): PublicOriginIdentityV1[] { + ensureLabDirs(configDir); + const dir = labPublicOriginDir(configDir); + const names = originNames(dir); + if (names.length > MAX_ORIGINS) { + throw new PublicEvidenceValidationError("public_origin_bound", "public origin marker bound exceeded"); + } + if (foreignOriginNames(dir).length > 0) { + throw new PublicEvidenceValidationError("public_origin_unsafe", "unexpected public origin marker entry"); + } + const identities: PublicOriginIdentityV1[] = []; + for (const name of names) { + const match = ORIGIN_RE.exec(name)!; + const expected = { publisherKeyId: match[1]!, bundleId: match[2]! }; + identities.push(readOrigin(join(dir, name), expected)); + } + return identities; +} + +export function clearLocalPublicOrigins(configDir?: string): void { + ensureLabDirs(configDir); + const dir = labPublicOriginDir(configDir); + cleanupStalePrivateFileStagesInDir(dir); + for (const name of readdirSync(dir)) { + if (!ORIGIN_RE.test(name)) continue; + try { unlinkSync(join(dir, name)); } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + } + } +} diff --git a/src/lab/public/privacy.ts b/src/lab/public/privacy.ts new file mode 100644 index 0000000000..f4af27a53e --- /dev/null +++ b/src/lab/public/privacy.ts @@ -0,0 +1,143 @@ +import { isIP } from "node:net"; +import type { + PublicArtifactV1, + PublicEvidenceBundleUnsignedV1, + PublicEvidenceBundleV1, + PublicEvidenceRecordV1, + PublicEvidenceSubjectV1, +} from "./types"; +import { PublicEvidenceValidationError } from "./validate"; + +const FORBIDDEN_PUBLIC_STRING_PATTERNS: ReadonlyArray<{ label: string; pattern: RegExp }> = [ + { label: "URL", pattern: /(?:https?|file):\/\//i }, + { + label: "local path", + pattern: + /(?:[A-Za-z]:[\\/]|\\\\[A-Za-z0-9._-]+\\|(?:^|[\s"'([{=:])\/(?:Users|home|root|tmp|var|opt|private|etc|mnt|media|srv|usr|dev|run|Library|System|Applications|Volumes|bin|sbin|lib|lib64|proc|sys|boot|Network|cores|nix|snap|app)(?:\/|$))/i, + }, + { label: "email", pattern: /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i }, + { label: "IP address", pattern: /\b(?:\d{1,3}\.){3}\d{1,3}\b|\[[0-9a-f:]{2,}\]/i }, + { label: "query string", pattern: /[?&][A-Za-z0-9_.~-]+=/ }, + { label: "authorization/header material", pattern: /\b(?:authorization|proxy-authorization|cookie|set-cookie|x-api-key|api[_-]?key|bearer)\b/i }, + { label: "credential", pattern: /\b(?:sk-[A-Za-z0-9_-]{8,}|gh[opusr]_[A-Za-z0-9]{8,}|github_pat_[A-Za-z0-9_]{8,}|AKIA[0-9A-Z]{12,})\b/ }, + { label: "private key", pattern: /-----BEGIN [^-]*PRIVATE KEY-----/i }, + { label: "local request/decision/Fabric id", pattern: /\b(?:request|decision|fabric)_[A-Za-z0-9_-]{6,}\b/i }, + { label: "account/project/tenant context", pattern: /\b(?:account|tenant|project|organization|deployment)[=:][^\s]+/i }, + { label: "precise timestamp", pattern: /\b\d{4}-\d{2}-\d{2}T\d{2}:\d{2}/ }, +]; + +const PUBLIC_TEXT_ARTIFACT_MEDIA_TYPES = new Set([ + "application/json", + "application/json; charset=utf-8", + "text/markdown", + "text/markdown; charset=utf-8", + "text/plain", + "text/plain; charset=utf-8", +]); + +function containsIpLiteral(value: string): boolean { + if (isIP(value) !== 0) return true; + for (const candidate of value.match(/[0-9A-Fa-f:]{2,}/g) ?? []) { + if (candidate.includes(":") && isIP(candidate) === 6) return true; + } + return false; +} + +function assertPrivacySafeString(value: string, field: string): void { + // `PUBLIC_IDENTIFIER` intentionally permits `:` for reviewed identifiers, so use + // Node's IP parser to validate colon-bearing candidates instead of rejecting them + // with a broad regex. This catches both whole-string and embedded IPv6 literals. + if (containsIpLiteral(value)) { + throw new PublicEvidenceValidationError( + "privacy_rejected", + `${field} contains forbidden IP address material`, + ); + } + for (const { label, pattern } of FORBIDDEN_PUBLIC_STRING_PATTERNS) { + if (pattern.test(value)) { + throw new PublicEvidenceValidationError( + "privacy_rejected", + `${field} contains forbidden ${label} material`, + ); + } + } +} + +function scanSubject(subject: PublicEvidenceSubjectV1, field: string): void { + if (subject.subjectKind === "protocol") { + assertPrivacySafeString(subject.compatibilityVersion, `${field}.compatibilityVersion`); + assertPrivacySafeString(subject.adapterFamily, `${field}.adapterFamily`); + assertPrivacySafeString(subject.inboundProtocol, `${field}.inboundProtocol`); + assertPrivacySafeString(subject.upstreamProtocol, `${field}.upstreamProtocol`); + assertPrivacySafeString(subject.surface, `${field}.surface`); + return; + } + if (subject.subjectKind === "route") { + assertPrivacySafeString(subject.providerId, `${field}.providerId`); + assertPrivacySafeString(subject.modelId, `${field}.modelId`); + assertPrivacySafeString(subject.adapterFamily, `${field}.adapterFamily`); + assertPrivacySafeString(subject.compatibilityVersion, `${field}.compatibilityVersion`); + return; + } + scanSubject(subject.route, `${field}.route`); + assertPrivacySafeString(subject.taskClassId, `${field}.taskClassId`); + assertPrivacySafeString(subject.taskClassVersion, `${field}.taskClassVersion`); + assertPrivacySafeString(subject.fabricCompatibilityVersion, `${field}.fabricCompatibilityVersion`); +} + +function scanArtifact(artifact: PublicArtifactV1, index: number): void { + const field = `bundle.artifacts[${index}]`; + assertPrivacySafeString(artifact.artifactClass, `${field}.artifactClass`); + assertPrivacySafeString(artifact.mediaType, `${field}.mediaType`); + + if (!PUBLIC_TEXT_ARTIFACT_MEDIA_TYPES.has(artifact.mediaType.toLowerCase())) { + throw new PublicEvidenceValidationError( + "privacy_rejected", + `${field}.mediaType is not in the closed public text-artifact set`, + ); + } + if (typeof artifact.contentBase64 !== "string") { + throw new PublicEvidenceValidationError("privacy_rejected", `${field}.contentBase64 is invalid`); + } + const bytes = Buffer.from(artifact.contentBase64, "base64"); + if (bytes.toString("base64") !== artifact.contentBase64 || bytes.byteLength !== artifact.byteCount) { + throw new PublicEvidenceValidationError( + "privacy_rejected", + `${field}.contentBase64 is non-canonical or does not match byteCount`, + ); + } + let text: string; + try { + text = new TextDecoder("utf-8", { fatal: true }).decode(bytes); + } catch { + throw new PublicEvidenceValidationError("privacy_rejected", `${field} is not valid UTF-8 text`); + } + assertPrivacySafeString(text, `${field}.content`); +} + +export function validatePublicEvidenceRecordPrivacy(record: PublicEvidenceRecordV1): void { + assertPrivacySafeString(record.suiteId, "record.suiteId"); + assertPrivacySafeString(record.suiteVersion, "record.suiteVersion"); + assertPrivacySafeString(record.scenarioId, "record.scenarioId"); + assertPrivacySafeString(record.scenarioVersion, "record.scenarioVersion"); + scanSubject(record.subject, "record.subject"); + for (const [index, assertion] of record.assertions.entries()) { + assertPrivacySafeString(assertion.id, `record.assertions[${index}].id`); + } + for (const [index, incident] of (record.incidentRefs ?? []).entries()) { + assertPrivacySafeString(incident.corpusId, `record.incidentRefs[${index}].corpusId`); + } +} + +/** + * Second-pass CL-10 export privacy boundary. Hashes, signatures and publisher public-key + * bytes are intentionally not pattern-scanned; every human-semantic public string and + * every final text artifact byte is scanned before local signing/storage or import. + */ +export function validatePublicEvidencePrivacy( + bundle: PublicEvidenceBundleUnsignedV1 | PublicEvidenceBundleV1, +): void { + assertPrivacySafeString(bundle.createdDayUtc, "bundle.createdDayUtc"); + for (const record of bundle.records) validatePublicEvidenceRecordPrivacy(record); + for (const [index, artifact] of bundle.artifacts.entries()) scanArtifact(artifact, index); +} diff --git a/src/lab/public/private-file.ts b/src/lab/public/private-file.ts new file mode 100644 index 0000000000..65e98325f1 --- /dev/null +++ b/src/lab/public/private-file.ts @@ -0,0 +1,261 @@ +import { randomUUID } from "node:crypto"; +import { + closeSync, + constants as fsConstants, + fsyncSync, + linkSync, + lstatSync, + openSync, + readdirSync, + unlinkSync, + writeSync, +} from "node:fs"; +import type { Stats } from "node:fs"; +import { basename, dirname, join } from "node:path"; + +export type PrivateFileCommitFault = "before_publish" | "parent_directory_sync" | null; +let privateFileCommitFaultForTests: PrivateFileCommitFault = null; +let privateFileCleanupSyncFaultForTests = false; +const PRIVATE_STAGE_RE = /^\..+\.(\d+)\.[0-9a-f-]{36}\.tmp$/; +export const PRIVATE_FILE_STAGE_RETENTION_MS = 24 * 60 * 60 * 1000; + +export interface PrivateFilePublishOptions { + /** Validate or harden the empty stage before caller-controlled bytes are written. */ + prepareStage?: (stagePath: string) => void; +} + +function cleanup(path: string): void { + try { unlinkSync(path); } catch { /* absent/already removed */ } +} + +function pidDefinitelyDead(pid: number): boolean { + try { + process.kill(pid, 0); + return false; + } catch (error) { + return (error as NodeJS.ErrnoException).code === "ESRCH"; + } +} + +function staleTempPrefix(finalPath: string): string { + return `.${basename(finalPath)}.`; +} + +function fsyncParentBestEffort(path: string): void { + if (process.platform === "win32") return; + let fd: number | null = null; + try { + fd = openSync(dirname(path), fsConstants.O_RDONLY); + fsyncSync(fd); + } catch { + // Cleanup durability is best-effort after unlink. Publication and crash-witness + // retirement use the strict path below and never swallow POSIX failures. + } finally { + if (fd !== null) closeSync(fd); + } +} + +function fsyncParentStrict(path: string): void { + if (process.platform === "win32") return; + let fd: number | null = null; + try { + fd = openSync(dirname(path), fsConstants.O_RDONLY); + fsyncSync(fd); + } catch (error) { + const code = (error as NodeJS.ErrnoException).code ?? "unknown"; + const wrapped = new Error(`private-file parent directory sync failed (${code})`); + (wrapped as Error & { cause?: unknown }).cause = error; + throw wrapped; + } finally { + if (fd !== null) closeSync(fd); + } +} + +function fsyncParentForPublication(path: string): void { + // Node does not provide a portable directory-fsync contract on Windows. The + // exclusive hard-link publication remains atomic there, while POSIX requires + // the parent directory sync before publication is reported as durable. + if (process.platform === "win32") return; + if (privateFileCommitFaultForTests === "parent_directory_sync") { + throw new Error("synthetic private-file parent directory sync failure"); + } + fsyncParentStrict(path); +} + +export function isPrivateFileStageName(name: string): boolean { + return PRIVATE_STAGE_RE.test(name); +} + +function isPrivateRegularStage(stats: Stats): boolean { + return stats.isFile() && !stats.isSymbolicLink(); +} + +function shouldReclaimPrivateFileStage(dir: string, name: string, nowMs: number): boolean { + const match = PRIVATE_STAGE_RE.exec(name); + if (!match) return false; + const pid = Number(match[1]); + if (!Number.isSafeInteger(pid) || pid <= 0) return false; + + let stats: Stats; + try { + stats = lstatSync(join(dir, name)); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return false; + throw error; + } + if (!isPrivateRegularStage(stats)) return false; + + const expired = nowMs - stats.mtimeMs > PRIVATE_FILE_STAGE_RETENTION_MS; + const dead = pid !== process.pid && pidDefinitelyDead(pid); + return expired || dead; +} + +/** Reclaim private-file stages whose writer is dead or whose crash witness is past the retention window. */ +export function cleanupStalePrivateFileStagesInDir(dir: string): void { + let names: string[]; + try { + names = readdirSync(dir); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return; + throw error; + } + const nowMs = Date.now(); + const reclaimable = names.filter((name) => shouldReclaimPrivateFileStage(dir, name, nowMs)); + if (reclaimable.length === 0) return; + + // A stale stage can be the hard-link witness for a final name that was linked + // before a crash or directory-sync failure. Make that final directory entry + // durable before removing any such witness. + if (process.platform !== "win32" && privateFileCleanupSyncFaultForTests) { + throw new Error("synthetic private-file cleanup parent directory sync failure"); + } + fsyncParentStrict(join(dir, ".")); + + let changed = false; + for (const name of reclaimable) { + try { + unlinkSync(join(dir, name)); + changed = true; + } catch { + // Another cleanup or writer may have removed it after enumeration. + } + } + if (changed) fsyncParentBestEffort(join(dir, ".")); +} + +/** Reclaim staging links from dead writers or expired crash witnesses. */ +export function cleanupStalePrivateFileStages(finalPath: string): void { + cleanupStalePrivateFileStagesInDir(dirname(finalPath)); +} + +/** Remove only stage links that already reference the durable final inode. */ +function cleanupPublishedPrivateFileStages(finalPath: string): void { + let finalStats; + try { + finalStats = lstatSync(finalPath); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return; + throw error; + } + if (!finalStats.isFile() || finalStats.isSymbolicLink()) return; + + const dir = dirname(finalPath); + const prefix = staleTempPrefix(finalPath); + let changed = false; + for (const name of readdirSync(dir)) { + if (!name.startsWith(prefix) || !PRIVATE_STAGE_RE.test(name)) continue; + const stagePath = join(dir, name); + try { + const stageStats = lstatSync(stagePath); + if (!stageStats.isFile() || stageStats.isSymbolicLink()) continue; + if (stageStats.dev !== finalStats.dev || stageStats.ino !== finalStats.ino) continue; + unlinkSync(stagePath); + changed = true; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + } + } + if (changed) fsyncParentBestEffort(finalPath); +} + +function writeAll(fd: number, bytes: Uint8Array): void { + let offset = 0; + while (offset < bytes.byteLength) { + const count = writeSync(fd, bytes, offset, bytes.byteLength - offset); + if (count <= 0) throw new Error("private file write made no progress"); + offset += count; + } +} + +/** + * Publish immutable mode-0600 bytes without ever exposing a partially-written final path. + * The caller owns EEXIST comparison semantics because some objects are idempotent and + * others are identity conflicts. Staging files are target-scoped and stale stages from + * definitely-dead writers are reclaimed on the next read or publication attempt. + */ +export function publishPrivateFileExclusive( + finalPath: string, + bytes: Uint8Array, + options: PrivateFilePublishOptions = {}, +): { created: boolean } { + cleanupStalePrivateFileStages(finalPath); + const tempPath = join( + dirname(finalPath), + `${staleTempPrefix(finalPath)}${process.pid}.${randomUUID()}.tmp`, + ); + let fd: number | null = null; + let preservePublishedStage = false; + try { + fd = openSync(tempPath, fsConstants.O_WRONLY | fsConstants.O_CREAT | fsConstants.O_EXCL, 0o600); + // Secret callers can harden the empty stage before any sensitive bytes exist. + // A failure here can therefore leave at most an empty cleanup witness. + options.prepareStage?.(tempPath); + writeAll(fd, bytes); + fsyncSync(fd); + closeSync(fd); + fd = null; + + if (privateFileCommitFaultForTests === "before_publish") { + throw new Error("synthetic private-file commit failure before publish"); + } + + try { + linkSync(tempPath, finalPath); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "EEXIST") { + // A prior publication may have linked the final entry but failed while + // syncing the parent directory. Re-sync before reporting idempotent success, + // then remove only stages that are hard links to that durable final inode. + fsyncParentForPublication(finalPath); + cleanupPublishedPrivateFileStages(finalPath); + return { created: false }; + } + throw error; + } + try { + fsyncParentForPublication(finalPath); + } catch (error) { + // The final name exists, but POSIX durability was not established. Keep this + // exact hard-link stage so a retry can re-sync and then identify it by inode. + preservePublishedStage = true; + throw error; + } + return { created: true }; + } finally { + if (fd !== null) closeSync(fd); + if (!preservePublishedStage) { + cleanup(tempPath); + fsyncParentBestEffort(finalPath); + } + } +} + +/** Test-only fault seam at the atomic publication point. Import this module directly in tests. */ +export function setPrivateFileCommitFaultForTests(fault: PrivateFileCommitFault): void { + privateFileCommitFaultForTests = fault; +} + +/** Test-only fault seam for strict stale-stage cleanup durability. */ +export function setPrivateFileCleanupSyncFaultForTests(enabled: boolean): void { + privateFileCleanupSyncFaultForTests = enabled; +} diff --git a/src/lab/public/project.ts b/src/lab/public/project.ts new file mode 100644 index 0000000000..77902d83d4 --- /dev/null +++ b/src/lab/public/project.ts @@ -0,0 +1,124 @@ +import type { CompatibilityVerdict } from "../constants"; +import type { ObservationEvent, ProtocolSubjectV1 } from "../events/types"; +import { validatePublicEvidenceAuthorities } from "./community-authority"; +import { publicEvidenceId } from "./ids"; +import { validatePublicEvidenceRecordPrivacy } from "./privacy"; +import { publicUtcDay } from "./time"; +import { + PUBLIC_ADAPTER_FAMILIES, + type PublicAdapterFamily, + type PublicEvidenceProjectionResult, + type PublicEvidenceRecordV1, + type PublicIncidentRefV1, + type PublicProtocolSubjectV1, +} from "./types"; +import { + isPublicIncidentRef, + PublicEvidenceValidationError, + validatePublicEvidenceRecord, +} from "./validate"; + +const PROJECTOR_INVARIANT_ERROR_CODES = new Set([ + "subject_id_mismatch", + "record_id_mismatch", +]); + +export interface ProjectPublicEvidenceRecordInput { + observation: ObservationEvent; + verdict: CompatibilityVerdict; + incidentRefs?: string[]; + publicArtifactRefs?: string[]; +} + +function asPublicAdapterFamily(value: string): PublicAdapterFamily | undefined { + return (PUBLIC_ADAPTER_FAMILIES as readonly string[]).includes(value) + ? value as PublicAdapterFamily + : undefined; +} + +function projectProtocolSubject(subject: ProtocolSubjectV1): PublicProtocolSubjectV1 | undefined { + const adapterFamily = asPublicAdapterFamily(subject.effectiveAdapter); + if (!adapterFamily) return undefined; + return { + subjectKind: "protocol", + compatibilityVersion: subject.opencodexCompatibilityVersion, + adapterFamily, + inboundProtocol: subject.inboundProtocol, + upstreamProtocol: subject.upstreamProtocol, + surface: subject.surface, + }; +} + +function projectIncidentRefs(values: string[] | undefined): PublicIncidentRefV1[] | undefined { + if (values === undefined) return undefined; + if (values.some((value) => !isPublicIncidentRef(value))) return undefined; + return values.map((corpusId) => ({ corpusId })); +} + +/** + * Project one local observation into the closed public V1 record shape and apply the + * complete reviewed authority/privacy boundary before exposing it as exportable. + * + * Route and task observations deliberately fail closed here. Persisted RouteSubjectV1 + * contains installation-salted provider-instance and endpoint identity, so the exact + * public/default route cannot be proven from ledger bytes alone. Dropping those fields + * would broaden a private exact route into a misleading public claim. + */ +export function projectPublicEvidenceRecord( + input: ProjectPublicEvidenceRecordInput, +): PublicEvidenceProjectionResult { + const { observation } = input; + + if (observation.evidenceLayer === "live_route_compatibility" || observation.evidenceLayer === "task_effectiveness") { + return { status: "not_exportable", reason: "private_route_identity" }; + } + if (observation.evidenceLayer !== "protocol_conformance" || observation.subject.subjectKind !== "protocol") { + return { status: "not_exportable", reason: "unsupported_subject" }; + } + + const subject = projectProtocolSubject(observation.subject); + if (!subject) return { status: "not_exportable", reason: "unsupported_adapter_family" }; + + const incidentRefs = projectIncidentRefs(input.incidentRefs); + if (input.incidentRefs !== undefined && incidentRefs === undefined) { + return { status: "not_exportable", reason: "unsafe_public_field" }; + } + + try { + const subjectId = publicEvidenceId("subject", subject); + const withoutRecordId: Omit<PublicEvidenceRecordV1, "recordId"> = { + subjectId, + evidenceLayer: "protocol_conformance", + suiteId: observation.suiteId, + suiteVersion: observation.suiteVersion, + scenarioId: observation.scenarioId, + scenarioVersion: observation.scenarioVersion, + verdict: input.verdict, + observedDayUtc: publicUtcDay(observation.completedAt), + subject, + assertions: observation.assertions.map((assertion) => ({ + id: assertion.id, + required: assertion.required, + passed: assertion.passed, + })), + ...(incidentRefs !== undefined ? { incidentRefs } : {}), + ...(input.publicArtifactRefs !== undefined ? { artifactRefs: [...input.publicArtifactRefs] } : {}), + }; + const record = validatePublicEvidenceRecord({ + recordId: publicEvidenceId("record", withoutRecordId), + ...withoutRecordId, + }); + validatePublicEvidenceAuthorities([record]); + validatePublicEvidenceRecordPrivacy(record); + return { status: "exportable", record }; + } catch (error) { + if (error instanceof PublicEvidenceValidationError) { + if (PROJECTOR_INVARIANT_ERROR_CODES.has(error.code)) throw error; + return { status: "not_exportable", reason: "unsafe_public_field", detailCode: error.code }; + } + if (error instanceof TypeError && error.message.startsWith("jcsStringify:")) { + return { status: "not_exportable", reason: "unsafe_public_field" }; + } + throw error; + } +} diff --git a/src/lab/public/purge-test-fault.ts b/src/lab/public/purge-test-fault.ts new file mode 100644 index 0000000000..f046b7594e --- /dev/null +++ b/src/lab/public/purge-test-fault.ts @@ -0,0 +1,21 @@ +export type PublicEvidencePurgeFaultForTests = + | "before_export_delete" + | "export_directory_sync" + | null; + +let purgeFaultForTests: PublicEvidencePurgeFaultForTests = null; + +/** Arms the internal deterministic fault seam and returns a scoped restore handle. */ +export function setPublicEvidencePurgeFaultForTests( + fault: PublicEvidencePurgeFaultForTests, +): () => void { + const previous = purgeFaultForTests; + purgeFaultForTests = fault; + return () => { + purgeFaultForTests = previous; + }; +} + +export function publicEvidencePurgeFaultForTests(): PublicEvidencePurgeFaultForTests { + return purgeFaultForTests; +} diff --git a/src/lab/public/purge.ts b/src/lab/public/purge.ts new file mode 100644 index 0000000000..01a063496f --- /dev/null +++ b/src/lab/public/purge.ts @@ -0,0 +1,223 @@ +import { createPrivateKey, createPublicKey } from "node:crypto"; +import { + closeSync, + constants as fsConstants, + fsyncSync, + openSync, + readdirSync, + rmSync, + unlinkSync, +} from "node:fs"; +import { join } from "node:path"; +import { + ensureLabDirs, + labCommunityDir, + labExportDir, + labPublicOriginDir, + labPublicPublisherKeyPath, +} from "../paths"; +import { + isCommunityRevocationFileName, + parseCommunityBundleFileName, +} from "./community-files"; +import { readPrivateRegularFile } from "./file-safety"; +import { publicEvidenceId } from "./ids"; +import { withPublicEvidenceMutationLock } from "./mutation-lock"; +import { clearLocalPublicOrigins } from "./origin"; +import { recoverPublicOriginsForPurge } from "./origin-purge"; +import { publicEvidencePurgeFaultForTests } from "./purge-test-fault"; +import { readPublicEvidenceBundle } from "./storage"; +import { parseStrictPublicJson } from "./strict-json"; +import { PublicEvidenceValidationError } from "./validate"; + +const MAX_PRIVATE_KEY_BYTES = 8 * 1024; +const MAX_COMMUNITY_OBJECT_BYTES = 2 * 1024 * 1024; +const EXPORT_FILE_RE = /^([0-9a-f]{64})\.json$/; + +function syncPurgeDirectory(dir: string, label: "export" | "community" | "origin"): void { + if (process.platform === "win32") return; + if (label === "export" && publicEvidencePurgeFaultForTests() === "export_directory_sync") { + throw new Error("synthetic public export directory sync failure"); + } + let fd: number | null = null; + try { + fd = openSync(dir, fsConstants.O_RDONLY); + fsyncSync(fd); + } catch (error) { + const detail = error instanceof Error ? error.message : String(error); + throw new Error(`public ${label} purge directory sync failed: ${detail}`); + } finally { + if (fd !== null) closeSync(fd); + } +} + +/** + * Publisher provenance is useful only for classifying local community copies. A corrupt + * key must never block deletion of sensitive exports, so classification fails closed to + * "unknown publisher" while the purge continues. + */ +function readExistingPublisherKeyId(configDir?: string): string | null { + const path = labPublicPublisherKeyPath(configDir); + try { + const pem = readPrivateRegularFile(path, { + maxBytes: MAX_PRIVATE_KEY_BYTES, + errorCode: "public_publisher_key_unsafe", + errorMessage: "public publisher key is unsafe during purge", + requireMode600: true, + }).toString("utf8"); + if (!pem.includes("BEGIN PRIVATE KEY")) return null; + const privateKey = createPrivateKey(pem); + if (privateKey.asymmetricKeyType !== "ed25519") return null; + const publicKey = createPublicKey(pem); + const publicKeyDer = publicKey.export({ type: "spki", format: "der" }).toString("base64"); + return publicEvidenceId("publisher_key", { algorithm: "ed25519", publicKey: publicKeyDer }); + } catch { + return null; + } +} + +function publicIdentity(publisherKeyId: string, bundleId: string): string { + return `${publisherKeyId}:${bundleId}`; +} + +/** Best-effort legacy classification only. Malformed exports are still deleted below. */ +function localExportIdentities(configDir?: string): Set<string> { + const identities = new Set<string>(); + for (const entry of readdirSync(labExportDir(configDir), { withFileTypes: true })) { + const match = EXPORT_FILE_RE.exec(entry.name); + if (!match) continue; + try { + const bundle = readPublicEvidenceBundle(match[1]!, configDir); + identities.add(publicIdentity(bundle.publisher.keyId, bundle.bundleId)); + } catch { + // Durable origin markers are the primary provenance source. Never retain a + // malformed export merely because legacy recovery can no longer parse it. + } + } + return identities; +} + +function purgeAllExports(configDir?: string): number { + if (publicEvidencePurgeFaultForTests() === "before_export_delete") { + throw new Error("synthetic public export purge failure"); + } + let deleted = 0; + const exportDir = labExportDir(configDir); + for (const entry of readdirSync(exportDir, { withFileTypes: true })) { + rmSync(join(exportDir, entry.name), { recursive: entry.isDirectory(), force: true }); + deleted += 1; + } + // A previous attempt may already have removed all names but failed its directory + // fsync. Re-sync even when this retry deletes zero entries before reporting success. + syncPurgeDirectory(exportDir, "export"); + return deleted; +} + +/** + * Provenance classification happens before this call. Purge removes only the exact + * community cache pathname, so a symlink or hardlink cannot redirect deletion to a peer. + */ +function unlinkLocalCommunityFile(path: string): boolean { + try { + unlinkSync(path); + return true; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return false; + throw error; + } +} + +function communityObjectPublisherKeyId(path: string): string | null { + try { + const raw = parseStrictPublicJson( + readPrivateRegularFile(path, { + maxBytes: MAX_COMMUNITY_OBJECT_BYTES, + errorCode: "community_unsafe_target", + errorMessage: "community object is unsafe during purge", + }), + "community object during purge", + ); + if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null; + const publisher = (raw as { publisher?: unknown }).publisher; + if (!publisher || typeof publisher !== "object" || Array.isArray(publisher)) return null; + const keyId = (publisher as { keyId?: unknown }).keyId; + return typeof keyId === "string" && /^[0-9a-f]{64}$/.test(keyId) ? keyId : null; + } catch { + return null; + } +} + +function purgeLocalPublicEvidenceCopiesLocked(configDir?: string): { + deletedExports: number; + deletedCommunityBundles: number; + deletedCommunityRevocations: number; +} { + const exportedIdentities = localExportIdentities(configDir); + const originRecovery = recoverPublicOriginsForPurge(configDir); + const localPublisherKeyIds = new Set<string>(); + for (const origin of originRecovery.identities) { + exportedIdentities.add(publicIdentity(origin.publisherKeyId, origin.bundleId)); + localPublisherKeyIds.add(origin.publisherKeyId); + } + const currentPublisherKeyId = readExistingPublisherKeyId(configDir); + if (currentPublisherKeyId) localPublisherKeyIds.add(currentPublisherKeyId); + const communityDir = labCommunityDir(configDir); + + // Sensitive local exports are the mandatory deletion target. Provenance is captured + // before this point, so cleanup remains possible even after the export bytes disappear. + const deletedExports = purgeAllExports(configDir); + + let deletedCommunityBundles = 0; + let deletedCommunityRevocations = 0; + for (const entry of readdirSync(communityDir, { withFileTypes: true })) { + const bundleIdentity = parseCommunityBundleFileName(entry.name); + if (bundleIdentity) { + const { publisherKeyId, bundleId } = bundleIdentity; + const locallyOriginated = exportedIdentities.has(publicIdentity(publisherKeyId, bundleId)) + || localPublisherKeyIds.has(publisherKeyId); + if (locallyOriginated && unlinkLocalCommunityFile(join(communityDir, entry.name))) { + deletedCommunityBundles += 1; + } + continue; + } + + if (isCommunityRevocationFileName(entry.name)) { + const path = join(communityDir, entry.name); + const publisherKeyId = communityObjectPublisherKeyId(path); + if (publisherKeyId && localPublisherKeyIds.has(publisherKeyId) + && unlinkLocalCommunityFile(path)) { + deletedCommunityRevocations += 1; + } + } + } + // As with exports, a retry after a failed directory fsync may have no remaining + // names to unlink. Re-sync the directory unconditionally before success. + syncPurgeDirectory(communityDir, "community"); + + if (originRecovery.skipped > 0) { + // Preserve provenance markers for operator recovery. Sensitive exports are already + // durably gone, but unknown community copies cannot be reported as fully purged. + throw new PublicEvidenceValidationError( + "public_origin_incomplete", + `public origin classification incomplete: ${originRecovery.skipped} marker(s) could not be validated`, + ); + } + + // Markers are purge-owned public provenance only. Remove them last, then establish + // deletion durability before the caller may record an export purge tombstone. + clearLocalPublicOrigins(configDir); + syncPurgeDirectory(labPublicOriginDir(configDir), "origin"); + return { deletedExports, deletedCommunityBundles, deletedCommunityRevocations }; +} + +export function purgeLocalPublicEvidenceCopies(configDir?: string): { + deletedExports: number; + deletedCommunityBundles: number; + deletedCommunityRevocations: number; +} { + ensureLabDirs(configDir); + return withPublicEvidenceMutationLock( + configDir, + () => purgeLocalPublicEvidenceCopiesLocked(configDir), + ); +} diff --git a/src/lab/public/registry.ts b/src/lab/public/registry.ts new file mode 100644 index 0000000000..40372a4fe2 --- /dev/null +++ b/src/lab/public/registry.ts @@ -0,0 +1,44 @@ +import { publicEvidenceId } from "./ids"; +import { + PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION, + type PublicAdapterFamily, + type PublicRouteRegistryEntryV1, + type PublicRouteRegistryManifestV1, +} from "./types"; + +// Repository-authoritative provider/model/adapter snapshot. The public manifest +// itself is independently content-addressed by manifestDigest below. +const PUBLIC_ROUTE_REGISTRY_SOURCE_COMMIT = "75a21417657ba5a3033198be0d8ae949de723d11"; + +const entries: PublicRouteRegistryEntryV1[] = [ + { + providerId: "openai", + modelId: "gpt-5.6-sol", + adapterFamilies: ["openai-responses"], + }, +]; + +const manifestWithoutDigest = { + schemaVersion: PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION, + registryVersion: "2026-08-13.v2", + sourceCommit: PUBLIC_ROUTE_REGISTRY_SOURCE_COMMIT, + entries, +}; + +export const PUBLIC_ROUTE_REGISTRY_V1: PublicRouteRegistryManifestV1 = Object.freeze({ + ...manifestWithoutDigest, + entries: Object.freeze(entries.map((entry) => Object.freeze({ + ...entry, + adapterFamilies: Object.freeze([...entry.adapterFamilies]) as unknown as PublicAdapterFamily[], + }))) as unknown as PublicRouteRegistryEntryV1[], + manifestDigest: publicEvidenceId("route_registry", manifestWithoutDigest), +}); + +export function findPublicRouteRegistryEntry( + providerId: string, + modelId: string, +): PublicRouteRegistryEntryV1 | undefined { + return PUBLIC_ROUTE_REGISTRY_V1.entries.find( + (entry) => entry.providerId === providerId && entry.modelId === modelId, + ); +} diff --git a/src/lab/public/revocation.ts b/src/lab/public/revocation.ts new file mode 100644 index 0000000000..58fdd0efb8 --- /dev/null +++ b/src/lab/public/revocation.ts @@ -0,0 +1,252 @@ +import { createPublicKey, verify as verifyBytes } from "node:crypto"; +import { publicEvidenceId } from "./ids"; +import { + loadExistingPublicPublisher, + signPublicPublisherDigest, + verifyPublicEvidenceBundle, +} from "./signature"; +import { + PUBLIC_EVIDENCE_REVOCATION_SCHEMA_VERSION, + type PublicEvidenceBundleV1, + type PublicEvidenceRevocationV1, + type PublicPublisherV1, + type PublicRevocationReasonV1, + type PublicRevocationTargetV1, + type PublicRevocationVerificationResult, +} from "./types"; +import { PublicEvidenceValidationError } from "./validate"; + +const REASONS = new Set<PublicRevocationReasonV1>([ + "publisher_retracted", + "privacy_retraction", + "evidence_invalidated", + "superseded", +]); +const MAX_TARGETS = 256; + +function isPlainObject(value: unknown): value is Record<string, unknown> { + return !!value && typeof value === "object" && !Array.isArray(value); +} + +function closedKeys(value: Record<string, unknown>, keys: readonly string[]): boolean { + const allowed = new Set(keys); + return Object.keys(value).every((key) => allowed.has(key)) && keys.every((key) => key in value); +} + +function validId(value: unknown): value is string { + return typeof value === "string" && /^[0-9a-f]{64}$/.test(value); +} + +function validDay(value: unknown): value is string { + if (typeof value !== "string" || !/^\d{4}-\d{2}-\d{2}$/.test(value)) return false; + const date = new Date(`${value}T00:00:00.000Z`); + return Number.isFinite(date.getTime()) && date.toISOString().slice(0, 10) === value; +} + +function targetKey(target: PublicRevocationTargetV1): string { + return `${target.kind}:${target.id}`; +} + +function compareCanonicalText(a: string, b: string): number { + if (a < b) return -1; + if (a > b) return 1; + return 0; +} + +function canonicalTargets(targets: readonly PublicRevocationTargetV1[]): PublicRevocationTargetV1[] { + if (targets.length === 0 || targets.length > MAX_TARGETS) { + throw new PublicEvidenceValidationError("revocation_targets", "revocation must contain 1..256 targets"); + } + const normalized = targets.map((target) => { + if ((target.kind !== "bundle" && target.kind !== "record") || !validId(target.id)) { + throw new PublicEvidenceValidationError("revocation_target", "invalid revocation target"); + } + return { kind: target.kind, id: target.id } as PublicRevocationTargetV1; + }).sort((a, b) => compareCanonicalText(targetKey(a), targetKey(b))); + if (new Set(normalized.map(targetKey)).size !== normalized.length) { + throw new PublicEvidenceValidationError("revocation_target_duplicate", "revocation targets must be unique"); + } + return normalized; +} + +function samePublisher(a: PublicPublisherV1, b: PublicPublisherV1): boolean { + return a.algorithm === b.algorithm && a.keyId === b.keyId && a.publicKey === b.publicKey; +} + +function validateTargetsAgainstBundle(targets: readonly PublicRevocationTargetV1[], bundle: PublicEvidenceBundleV1): boolean { + const records = new Set(bundle.records.map((record) => record.recordId)); + return targets.every((target) => target.kind === "bundle" ? target.id === bundle.bundleId : records.has(target.id)); +} + +function revocationPayload( + issuedDayUtc: string, + publisher: PublicPublisherV1, + targets: PublicRevocationTargetV1[], + reason: PublicRevocationReasonV1, +): Record<string, unknown> { + return { + schemaVersion: PUBLIC_EVIDENCE_REVOCATION_SCHEMA_VERSION, + issuedDayUtc, + publisher, + targets, + reason, + }; +} + +export function createPublicEvidenceRevocation(input: { + configDir?: string; + targetBundle: PublicEvidenceBundleV1; + issuedDayUtc: string; + targets: PublicRevocationTargetV1[]; + reason: PublicRevocationReasonV1; +}): PublicEvidenceRevocationV1 { + // Validate the target and every caller-controlled field before touching publisher state. + if (verifyPublicEvidenceBundle(input.targetBundle).status !== "cryptographically_valid") { + throw new PublicEvidenceValidationError("revocation_target", "revocation target bundle is not cryptographically valid"); + } + if (!validDay(input.issuedDayUtc)) { + throw new PublicEvidenceValidationError("revocation_day", "issuedDayUtc is invalid"); + } + if (!REASONS.has(input.reason)) { + throw new PublicEvidenceValidationError("revocation_reason", "unsupported revocation reason"); + } + const targets = canonicalTargets(input.targets); + if (!validateTargetsAgainstBundle(targets, input.targetBundle)) { + throw new PublicEvidenceValidationError("revocation_target", "revocation target is unknown to target bundle"); + } + + const handle = loadExistingPublicPublisher(input.configDir); + if (!handle || !samePublisher(handle.publisher, input.targetBundle.publisher)) { + throw new PublicEvidenceValidationError( + "revocation_publisher", + "revocation requires the existing publisher key that signed the target bundle", + ); + } + const revocationId = publicEvidenceId( + "revocation", + revocationPayload(input.issuedDayUtc, handle.publisher, targets, input.reason), + ); + const signature = signPublicPublisherDigest(handle, revocationId); + return Object.freeze({ + schemaVersion: PUBLIC_EVIDENCE_REVOCATION_SCHEMA_VERSION, + revocationId, + issuedDayUtc: input.issuedDayUtc, + publisher: handle.publisher, + targets, + reason: input.reason, + signature: Object.freeze({ algorithm: "ed25519" as const, signedDigest: revocationId, signature }), + }); +} + +export function verifyPublicEvidenceRevocation( + raw: unknown, + targetBundle: PublicEvidenceBundleV1, +): PublicRevocationVerificationResult { + try { + if (!isPlainObject(raw) || !closedKeys(raw, [ + "schemaVersion", "revocationId", "issuedDayUtc", "publisher", "targets", "reason", "signature", + ])) { + return { status: "schema_rejected", detail: "closed revocation schema mismatch" }; + } + if ( + raw.schemaVersion !== PUBLIC_EVIDENCE_REVOCATION_SCHEMA_VERSION + || !validId(raw.revocationId) + || !validDay(raw.issuedDayUtc) + || !REASONS.has(raw.reason as PublicRevocationReasonV1) + ) { + return { status: "schema_rejected", detail: "revocation version/id/day/reason invalid" }; + } + if ( + !isPlainObject(raw.publisher) + || !closedKeys(raw.publisher, ["algorithm", "keyId", "publicKey"]) + || raw.publisher.algorithm !== "ed25519" + || !validId(raw.publisher.keyId) + || typeof raw.publisher.publicKey !== "string" + || raw.publisher.publicKey.length > 1024 + ) { + return { status: "schema_rejected", detail: "revocation publisher invalid" }; + } + const publicKeyBytes = Buffer.from(raw.publisher.publicKey, "base64"); + if (publicKeyBytes.toString("base64") !== raw.publisher.publicKey) { + return { status: "schema_rejected", detail: "revocation publisher key is non-canonical" }; + } + const publisher: PublicPublisherV1 = { + algorithm: "ed25519", + keyId: raw.publisher.keyId, + publicKey: raw.publisher.publicKey, + }; + if (publicEvidenceId("publisher_key", { algorithm: "ed25519", publicKey: publisher.publicKey }) !== publisher.keyId) { + return { status: "schema_rejected", detail: "revocation publisher key id mismatch" }; + } + if (!samePublisher(publisher, targetBundle.publisher)) { + return { status: "publisher_mismatch", detail: "revocation publisher does not match target bundle" }; + } + if (!Array.isArray(raw.targets) || raw.targets.length === 0 || raw.targets.length > MAX_TARGETS) { + return { status: "schema_rejected", detail: "revocation targets invalid" }; + } + const targets: PublicRevocationTargetV1[] = []; + for (const [index, value] of raw.targets.entries()) { + if ( + !isPlainObject(value) + || !closedKeys(value, ["kind", "id"]) + || (value.kind !== "bundle" && value.kind !== "record") + || !validId(value.id) + ) { + return { status: "schema_rejected", detail: `revocation target ${index} invalid` }; + } + targets.push({ kind: value.kind, id: value.id }); + } + const canonical = canonicalTargets(targets); + if (canonical.some((target, index) => target.kind !== targets[index]!.kind || target.id !== targets[index]!.id)) { + return { status: "schema_rejected", detail: "revocation targets must be sorted" }; + } + if (!validateTargetsAgainstBundle(targets, targetBundle)) { + return { status: "unknown_target", detail: "revocation target not present in target bundle" }; + } + if ( + !isPlainObject(raw.signature) + || !closedKeys(raw.signature, ["algorithm", "signedDigest", "signature"]) + || raw.signature.algorithm !== "ed25519" + || raw.signature.signedDigest !== raw.revocationId + || typeof raw.signature.signature !== "string" + ) { + return { status: "schema_rejected", detail: "revocation signature schema invalid" }; + } + const expected = publicEvidenceId( + "revocation", + revocationPayload(raw.issuedDayUtc, publisher, targets, raw.reason as PublicRevocationReasonV1), + ); + if (expected !== raw.revocationId) { + return { status: "digest_invalid", detail: "revocation id does not match canonical bytes" }; + } + const key = createPublicKey({ key: publicKeyBytes, type: "spki", format: "der" }); + if (key.asymmetricKeyType !== "ed25519") { + return { status: "signature_invalid", detail: "revocation publisher key is not Ed25519" }; + } + const signatureBytes = Buffer.from(raw.signature.signature, "base64"); + if (signatureBytes.toString("base64") !== raw.signature.signature) { + return { status: "signature_invalid", detail: "revocation signature is non-canonical" }; + } + if (!verifyBytes(null, Buffer.from(raw.revocationId, "hex"), key, signatureBytes)) { + return { status: "signature_invalid", detail: "revocation signature invalid" }; + } + return { + status: "cryptographically_valid", + revocation: { + schemaVersion: PUBLIC_EVIDENCE_REVOCATION_SCHEMA_VERSION, + revocationId: raw.revocationId, + issuedDayUtc: raw.issuedDayUtc, + publisher, + targets, + reason: raw.reason as PublicRevocationReasonV1, + signature: { + algorithm: "ed25519", + signedDigest: raw.revocationId, + signature: raw.signature.signature, + }, + }, + }; + } catch (error) { + return { status: "schema_rejected", detail: error instanceof Error ? error.message : String(error) }; + } +} diff --git a/src/lab/public/signature.ts b/src/lab/public/signature.ts new file mode 100644 index 0000000000..c88c903a0d --- /dev/null +++ b/src/lab/public/signature.ts @@ -0,0 +1,219 @@ +import { + createPrivateKey, + createPublicKey, + generateKeyPairSync, + sign as signBytes, + verify as verifyBytes, +} from "node:crypto"; +import { ensureLabDirs, labPublicPublisherKeyPath } from "../paths"; +import { hardenSecretPath } from "../../lib/windows-secret-acl"; +import { + buildPublicEvidenceBundle, + canonicalPublicEvidenceContent, + expectedPublicBundleIdentityFromNormalized, + normalizePublicEvidenceContent, + type BuildPublicEvidenceBundleInput, +} from "./bundle"; +import { validatePublicEvidenceAuthorities } from "./community-authority"; +import { privateRegularFileSize, readPrivateRegularFile } from "./file-safety"; +import { publicEvidenceId } from "./ids"; +import { cleanupStalePrivateFileStages, publishPrivateFileExclusive } from "./private-file"; +import { validatePublicEvidencePrivacy, validatePublicEvidenceRecordPrivacy } from "./privacy"; +import type { + PublicEvidenceBundleV1, + PublicPublisherV1, +} from "./types"; +import { PublicEvidenceValidationError } from "./validate"; + +const MAX_PRIVATE_KEY_BYTES = 8 * 1024; +const PRIVATE_KEY_FILE_OPTIONS = { + maxBytes: MAX_PRIVATE_KEY_BYTES, + errorCode: "public_publisher_key_unsafe", + errorMessage: "public publisher key path is not a bounded private regular file with 0600 permissions", + requireMode600: true, +} as const; + +export interface PublicPublisherHandle { + publisher: PublicPublisherV1; + privateKeyPath: string; +} + +function publicKeyBase64(privateKeyPem: string): string { + const publicKey = createPublicKey(privateKeyPem); + return publicKey.export({ type: "spki", format: "der" }).toString("base64"); +} + +function publisherForPrivateKey(privateKeyPem: string): PublicPublisherV1 { + const publicKey = publicKeyBase64(privateKeyPem); + return { + algorithm: "ed25519", + keyId: publicEvidenceId("publisher_key", { algorithm: "ed25519", publicKey }), + publicKey, + }; +} + +function requirePublisherKeyAcl(path: string, timeoutMemoKey = path): void { + privateRegularFileSize(path, PRIVATE_KEY_FILE_OPTIONS); + let hardened: { ok: boolean }; + let hardeningError: unknown; + try { + hardened = hardenSecretPath(path, { required: true, timeoutMemoKey }); + } catch (error) { + hardeningError = error; + hardened = { ok: false }; + } + if (!hardened.ok) { + const failure = new PublicEvidenceValidationError( + "public_publisher_key_unsafe", + "public publisher key ACL hardening did not complete", + ); + if (hardeningError !== undefined) { + (failure as Error & { cause?: unknown }).cause = hardeningError; + } + throw failure; + } +} + +function readRestrictedPrivateKey(path: string): string { + cleanupStalePrivateFileStages(path); + // Prove the pathname is the expected private regular file before applying any + // platform ACL operation, then fail closed if Windows per-user ACL hardening + // cannot be established. The helper is a no-op success on non-Windows. + requirePublisherKeyAcl(path); + const pem = readPrivateRegularFile(path, PRIVATE_KEY_FILE_OPTIONS).toString("utf8"); + const key = createPrivateKey(pem); + if (key.asymmetricKeyType !== "ed25519") { + throw new Error("public publisher key must be Ed25519"); + } + return pem; +} + +function createPrivateKeyFile(path: string): string { + const { privateKey } = generateKeyPairSync("ed25519", { + privateKeyEncoding: { type: "pkcs8", format: "pem" }, + publicKeyEncoding: { type: "spki", format: "pem" }, + }); + publishPrivateFileExclusive(path, Buffer.from(privateKey, "utf8"), { + // On Windows, harden the empty stage before private key bytes are written. + // A required ACL failure therefore cannot strand secret bytes in a stage. + prepareStage: stagePath => requirePublisherKeyAcl(stagePath, path), + }); + return readRestrictedPrivateKey(path); +} + +export function loadExistingPublicPublisher(configDir?: string): PublicPublisherHandle | null { + const privateKeyPath = labPublicPublisherKeyPath(configDir); + try { + const privateKeyPem = readRestrictedPrivateKey(privateKeyPath); + return { publisher: publisherForPrivateKey(privateKeyPem), privateKeyPath }; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return null; + throw error; + } +} + +export function getOrCreatePublicPublisher(configDir?: string): PublicPublisherHandle { + ensureLabDirs(configDir); + const existing = loadExistingPublicPublisher(configDir); + if (existing) return existing; + const privateKeyPath = labPublicPublisherKeyPath(configDir); + const privateKeyPem = createPrivateKeyFile(privateKeyPath); + return { publisher: publisherForPrivateKey(privateKeyPem), privateKeyPath }; +} + +/** Centralized descriptor-bound signing primitive for the installation publisher key. */ +export function signPublicPublisherDigest(handle: PublicPublisherHandle, digestHex: string): string { + if (!/^[0-9a-f]{64}$/.test(digestHex)) { + throw new PublicEvidenceValidationError("invalid_digest", "publisher signing digest must be lowercase sha256 hex"); + } + const privateKeyPem = readRestrictedPrivateKey(handle.privateKeyPath); + return signBytes(null, Buffer.from(digestHex, "hex"), createPrivateKey(privateKeyPem)).toString("base64"); +} + +export interface SignPublicEvidenceBundleInput extends Omit<BuildPublicEvidenceBundleInput, "publisher"> { + configDir?: string; +} + +function assertLocalArtifactExportAuthority(input: SignPublicEvidenceBundleInput): void { + if (input.artifacts.length !== 0) { + throw new PublicEvidenceValidationError( + "public_artifact_authority_required", + "artifact bytes require reviewed public_export policy authority before local signing", + ); + } +} + +export function signPublicEvidenceBundle(input: SignPublicEvidenceBundleInput): PublicEvidenceBundleV1 { + // Validate every caller-controlled invariant before publisher identity state is touched. + assertLocalArtifactExportAuthority(input); + const normalized = normalizePublicEvidenceContent({ + records: input.records, + artifacts: input.artifacts, + createdDayUtc: input.createdDayUtc, + }); + validatePublicEvidenceAuthorities(normalized.records); + for (const record of normalized.records) validatePublicEvidenceRecordPrivacy(record); + + const handle = getOrCreatePublicPublisher(input.configDir); + const unsigned = buildPublicEvidenceBundle({ ...normalized, publisher: handle.publisher }); + validatePublicEvidencePrivacy(unsigned); + return { + ...unsigned, + signature: { + algorithm: "ed25519", + signedDigest: unsigned.bundleDigest, + signature: signPublicPublisherDigest(handle, unsigned.bundleDigest), + }, + }; +} + +export type PublicBundleVerificationResult = + | { status: "cryptographically_valid" } + | { status: "digest_invalid" } + | { status: "signature_invalid" } + | { status: "schema_rejected" }; + +export function verifyPublicEvidenceBundle(bundle: PublicEvidenceBundleV1): PublicBundleVerificationResult { + try { + const raw = bundle as unknown as Record<string, unknown>; + if (!raw || typeof raw !== "object" || Array.isArray(raw)) return { status: "schema_rejected" }; + const allowed = new Set([ + "schemaVersion", + "exportPolicyVersion", + "bundleId", + "createdDayUtc", + "publisher", + "records", + "artifacts", + "bundleDigest", + "signature", + ]); + if (Object.keys(raw).some((key) => !allowed.has(key))) return { status: "schema_rejected" }; + if (bundle.schemaVersion !== "public_evidence_bundle_v1" || bundle.exportPolicyVersion !== "public_export_policy_v1") { + return { status: "schema_rejected" }; + } + if (!bundle.signature || bundle.signature.algorithm !== "ed25519") return { status: "schema_rejected" }; + if (Object.keys(bundle.signature).some((key) => !["algorithm", "signedDigest", "signature"].includes(key))) { + return { status: "schema_rejected" }; + } + const canonical = canonicalPublicEvidenceContent(bundle); + if (!canonical.canonical) return { status: "schema_rejected" }; + const expected = expectedPublicBundleIdentityFromNormalized(canonical.normalized, bundle.publisher); + if (bundle.bundleId !== expected.bundleId || bundle.bundleDigest !== expected.bundleDigest) { + return { status: "digest_invalid" }; + } + if (bundle.signature.signedDigest !== bundle.bundleDigest) return { status: "signature_invalid" }; + const key = createPublicKey({ + key: Buffer.from(bundle.publisher.publicKey, "base64"), + type: "spki", + format: "der", + }); + if (key.asymmetricKeyType !== "ed25519") return { status: "signature_invalid" }; + const signature = Buffer.from(bundle.signature.signature, "base64"); + if (signature.toString("base64") !== bundle.signature.signature) return { status: "signature_invalid" }; + const valid = verifyBytes(null, Buffer.from(bundle.bundleDigest, "hex"), key, signature); + return valid ? { status: "cryptographically_valid" } : { status: "signature_invalid" }; + } catch { + return { status: "schema_rejected" }; + } +} diff --git a/src/lab/public/storage.ts b/src/lab/public/storage.ts new file mode 100644 index 0000000000..8fb0a96179 --- /dev/null +++ b/src/lab/public/storage.ts @@ -0,0 +1,105 @@ +import { join } from "node:path"; +import { isSha256Hex, jcsStringify } from "../digest"; +import { ensureLabDirs } from "../paths"; +import { MAX_PUBLIC_BUNDLE_BYTES } from "./bundle"; +import { validatePublicEvidenceAuthorities } from "./community-authority"; +import { readPrivateRegularFile } from "./file-safety"; +import { cleanupStalePrivateFileStages, publishPrivateFileExclusive } from "./private-file"; +import { validatePublicEvidencePrivacy } from "./privacy"; +import { parseStrictPublicJson } from "./strict-json"; +import type { PublicEvidenceBundleV1 } from "./types"; +import { verifyPublicEvidenceBundle } from "./signature"; +import { PublicEvidenceValidationError } from "./validate"; + +function encodedBytes(value: string): number { + return new TextEncoder().encode(value).byteLength; +} + +function bundlePath(bundleId: string, configDir?: string): string { + if (!isSha256Hex(bundleId)) throw new Error("public bundle id must be lowercase sha256 hex"); + return join(ensureLabDirs(configDir).exportDir, `${bundleId}.json`); +} + +function assertLocalArtifactExportAuthority(bundle: PublicEvidenceBundleV1): void { + if (bundle.artifacts.length !== 0) { + throw new PublicEvidenceValidationError( + "public_artifact_authority_required", + "artifact bytes require reviewed public_export policy authority before local export storage", + ); + } +} + +function readLocalExport(path: string): Buffer { + cleanupStalePrivateFileStages(path); + return readPrivateRegularFile(path, { + maxBytes: MAX_PUBLIC_BUNDLE_BYTES, + errorCode: "public_file_unsafe", + errorMessage: "public export is not a private regular file with 0600 permissions", + sizeErrorCode: "public_file_too_large", + sizeErrorMessage: "public bundle exceeds 2 MiB", + requireMode600: true, + }); +} + +function existingBody(path: string): string | null { + try { + return readLocalExport(path).toString("utf8"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return null; + throw error; + } +} + +function validateLocalBundle(bundle: PublicEvidenceBundleV1): void { + const verification = verifyPublicEvidenceBundle(bundle); + if (verification.status !== "cryptographically_valid") { + throw new PublicEvidenceValidationError(verification.status, `public bundle verification failed: ${verification.status}`); + } + assertLocalArtifactExportAuthority(bundle); + validatePublicEvidenceAuthorities(bundle.records); + validatePublicEvidencePrivacy(bundle); +} + +export function storePublicEvidenceBundle( + bundle: PublicEvidenceBundleV1, + configDir?: string, +): { path: string; created: boolean } { + validateLocalBundle(bundle); + const body = jcsStringify(bundle) + "\n"; + if (encodedBytes(body) > MAX_PUBLIC_BUNDLE_BYTES) { + throw new PublicEvidenceValidationError("public_file_too_large", "public bundle exceeds 2 MiB"); + } + const path = bundlePath(bundle.bundleId, configDir); + const existing = existingBody(path); + if (existing !== null) { + if (existing === body) return { path, created: false }; + throw new PublicEvidenceValidationError("public_export_conflict", "public export id collision with different bytes"); + } + + const published = publishPrivateFileExclusive(path, Buffer.from(body, "utf8")); + if (!published.created) { + const raced = existingBody(path); + if (raced === body) return { path, created: false }; + throw new PublicEvidenceValidationError("public_export_conflict", "public export id collision with different bytes"); + } + return { path, created: true }; +} + +/** Backward-compatible local storage helper for callers that need the private path. */ +export function writePublicEvidenceBundle(bundle: PublicEvidenceBundleV1, configDir?: string): string { + return storePublicEvidenceBundle(bundle, configDir).path; +} + +export function readPublicEvidenceBundle(bundleId: string, configDir?: string): PublicEvidenceBundleV1 { + const bytes = readLocalExport(bundlePath(bundleId, configDir)); + const raw = parseStrictPublicJson(bytes, "public export", "public_file_json"); + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + throw new PublicEvidenceValidationError("public_file_json", "public export must contain a bundle object"); + } + const parsed = raw as PublicEvidenceBundleV1; + if (parsed.bundleId !== bundleId) { + throw new PublicEvidenceValidationError("public_file_identity", "public export filename does not match bundle id"); + } + validateLocalBundle(parsed); + return parsed; +} diff --git a/src/lab/public/strict-json.ts b/src/lab/public/strict-json.ts new file mode 100644 index 0000000000..339c41797c --- /dev/null +++ b/src/lab/public/strict-json.ts @@ -0,0 +1,206 @@ +import { PublicEvidenceValidationError } from "./validate"; + +const MAX_PUBLIC_JSON_BYTES = 2 * 1024 * 1024; +const MAX_PUBLIC_JSON_DEPTH = 8; +const MAX_PUBLIC_JSON_OBJECT_KEYS = 64; +const MAX_PUBLIC_JSON_ARRAY_ELEMENTS = 512; +const MAX_PUBLIC_JSON_STRING_BYTES = 384 * 1024; + +function isJsonWhitespace(value: string | undefined): boolean { + return value === " " || value === "\n" || value === "\r" || value === "\t"; +} + +function malformedJson(code: string, message: string): never { + throw new PublicEvidenceValidationError(code, message); +} + +function assertStrictPublicJsonShape(text: string, invalidCode: string): void { + let index = 0; + let depth = 0; + + function invalid(message: string): never { + return malformedJson(invalidCode, message); + } + + function skipWhitespace(): void { + while (isJsonWhitespace(text[index])) index += 1; + } + + function parseStringToken(): string { + if (text[index] !== '"') invalid("public JSON contains an invalid string token"); + const start = index; + index += 1; + let escaped = false; + while (index < text.length) { + const ch = text[index++]!; + if (escaped) { + escaped = false; + continue; + } + if (ch === "\\") { + escaped = true; + continue; + } + if (ch === '"') { + if (Buffer.byteLength(text.slice(start + 1, index - 1), "utf8") > MAX_PUBLIC_JSON_STRING_BYTES) { + invalid(`public JSON string exceeds ${MAX_PUBLIC_JSON_STRING_BYTES} bytes`); + } + try { + const decoded = JSON.parse(text.slice(start, index)); + if (typeof decoded !== "string") invalid("public JSON contains an invalid string token"); + return decoded; + } catch (error) { + if (error instanceof PublicEvidenceValidationError) throw error; + invalid("public JSON contains an invalid string token"); + } + } + if (ch.charCodeAt(0) < 0x20) invalid("public JSON contains an invalid control character"); + } + invalid("public JSON contains an unterminated string token"); + } + + function parseScalar(): void { + const start = index; + while (index < text.length) { + const ch = text[index]; + if (ch === "," || ch === "]" || ch === "}" || isJsonWhitespace(ch)) break; + index += 1; + } + if (start === index) invalid("public JSON contains an invalid value"); + try { + const parsed = JSON.parse(text.slice(start, index)); + if (parsed !== null && typeof parsed === "object") invalid("public JSON contains an invalid scalar value"); + } catch (error) { + if (error instanceof PublicEvidenceValidationError) throw error; + invalid("public JSON contains an invalid scalar value"); + } + } + + function enterContainer(): void { + depth += 1; + if (depth > MAX_PUBLIC_JSON_DEPTH) { + invalid(`public JSON nesting depth exceeds ${MAX_PUBLIC_JSON_DEPTH}`); + } + } + + function parseArray(): void { + enterContainer(); + try { + index += 1; + skipWhitespace(); + if (text[index] === "]") { + index += 1; + return; + } + let elementCount = 0; + while (index < text.length) { + elementCount += 1; + if (elementCount > MAX_PUBLIC_JSON_ARRAY_ELEMENTS) { + invalid(`public JSON array exceeds ${MAX_PUBLIC_JSON_ARRAY_ELEMENTS} elements`); + } + parseValue(); + skipWhitespace(); + if (text[index] === "]") { + index += 1; + return; + } + if (text[index] !== ",") invalid("public JSON array is malformed"); + index += 1; + skipWhitespace(); + if (text[index] === "]") invalid("public JSON array contains a trailing comma"); + } + invalid("public JSON array is unterminated"); + } finally { + depth -= 1; + } + } + + function parseObject(): void { + enterContainer(); + try { + index += 1; + skipWhitespace(); + if (text[index] === "}") { + index += 1; + return; + } + const keys = new Set<string>(); + while (index < text.length) { + if (text[index] !== '"') invalid("public JSON object key must be a string"); + const key = parseStringToken(); + if (keys.has(key)) { + throw new PublicEvidenceValidationError("duplicate_json_key", "duplicate JSON object key"); + } + keys.add(key); + if (keys.size > MAX_PUBLIC_JSON_OBJECT_KEYS) { + invalid(`public JSON object exceeds ${MAX_PUBLIC_JSON_OBJECT_KEYS} keys`); + } + skipWhitespace(); + if (text[index] !== ":") invalid("public JSON object is missing a colon"); + index += 1; + parseValue(); + skipWhitespace(); + if (text[index] === "}") { + index += 1; + return; + } + if (text[index] !== ",") invalid("public JSON object is malformed"); + index += 1; + skipWhitespace(); + if (text[index] === "}") invalid("public JSON object contains a trailing comma"); + } + invalid("public JSON object is unterminated"); + } finally { + depth -= 1; + } + } + + function parseValue(): void { + skipWhitespace(); + const ch = text[index]; + if (ch === "{") { + parseObject(); + return; + } + if (ch === "[") { + parseArray(); + return; + } + if (ch === '"') { + parseStringToken(); + return; + } + parseScalar(); + } + + skipWhitespace(); + if (index === text.length) invalid("public JSON is empty"); + parseValue(); + skipWhitespace(); + if (index !== text.length) invalid("public JSON contains trailing data"); +} + +export function parseStrictPublicJson( + bytes: Uint8Array, + label = "public JSON", + invalidCode = "public_json", + maxBytes = MAX_PUBLIC_JSON_BYTES, +): unknown { + if (!Number.isSafeInteger(maxBytes) || maxBytes < 1) { + throw new PublicEvidenceValidationError(invalidCode, `${label} byte limit is invalid`); + } + if (bytes.byteLength > maxBytes) { + throw new PublicEvidenceValidationError(invalidCode, `${label} exceeds ${maxBytes} bytes`); + } + const buffer = Buffer.from(bytes); + const text = buffer.toString("utf8"); + if (!Buffer.from(text, "utf8").equals(buffer)) { + throw new PublicEvidenceValidationError(invalidCode, `${label} is not valid UTF-8 JSON`); + } + assertStrictPublicJsonShape(text, invalidCode); + try { + return JSON.parse(text); + } catch { + throw new PublicEvidenceValidationError(invalidCode, `${label} is not valid JSON`); + } +} diff --git a/src/lab/public/time.ts b/src/lab/public/time.ts new file mode 100644 index 0000000000..3bc3fe284f --- /dev/null +++ b/src/lab/public/time.ts @@ -0,0 +1,26 @@ +import { PublicEvidenceValidationError } from "./validate"; + +/** Largest timestamp whose ISO-8601 year still fits the four-digit YYYY form. */ +const MAX_PUBLIC_DAY_TIMESTAMP_MS = Date.UTC(9999, 11, 31, 23, 59, 59, 999); + +/** Convert a bounded JavaScript timestamp into the public UTC day bucket. */ +export function publicUtcDay(timestampMs: number): string { + if ( + !Number.isInteger(timestampMs) + || timestampMs < 0 + || timestampMs > MAX_PUBLIC_DAY_TIMESTAMP_MS + ) { + throw new PublicEvidenceValidationError( + "public_selection_time", + "invalid observation completion timestamp", + ); + } + const date = new Date(timestampMs); + if (!Number.isFinite(date.getTime())) { + throw new PublicEvidenceValidationError( + "public_selection_time", + "invalid observation completion timestamp", + ); + } + return date.toISOString().slice(0, 10); +} diff --git a/src/lab/public/types.ts b/src/lab/public/types.ts new file mode 100644 index 0000000000..70d4b598fd --- /dev/null +++ b/src/lab/public/types.ts @@ -0,0 +1,172 @@ +import type { CompatibilityVerdict, EvidenceLayer } from "../constants"; + +export const PUBLIC_EVIDENCE_BUNDLE_SCHEMA_VERSION = "public_evidence_bundle_v1" as const; +export const PUBLIC_EXPORT_POLICY_VERSION = "public_export_policy_v1" as const; +export const PUBLIC_EVIDENCE_REVOCATION_SCHEMA_VERSION = "public_evidence_revocation_v1" as const; +export const PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION = "public_route_registry_v1" as const; + +export const PUBLIC_ADAPTER_FAMILIES = [ + "openai-responses", + "openai-chat", + "anthropic-messages", +] as const; +export type PublicAdapterFamily = (typeof PUBLIC_ADAPTER_FAMILIES)[number]; + +export interface PublicRouteRegistryEntryV1 { + providerId: string; + modelId: string; + adapterFamilies: PublicAdapterFamily[]; +} + +export interface PublicRouteRegistryManifestV1 { + schemaVersion: typeof PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION; + registryVersion: string; + sourceCommit: string; + entries: PublicRouteRegistryEntryV1[]; + manifestDigest: string; +} + +export interface PublicProtocolSubjectV1 { + subjectKind: "protocol"; + compatibilityVersion: string; + adapterFamily: PublicAdapterFamily; + inboundProtocol: string; + upstreamProtocol: string; + surface: string; +} + +export interface PublicRouteSubjectV1 { + subjectKind: "route"; + providerId: string; + modelId: string; + adapterFamily: PublicAdapterFamily; + compatibilityVersion: string; +} + +export interface PublicTaskSubjectV1 { + subjectKind: "task"; + route: PublicRouteSubjectV1; + taskClassId: string; + taskClassVersion: string; + taskFixtureDigest: string; + verifierManifestDigest: string; + fabricCompatibilityVersion: string; +} + +export type PublicEvidenceSubjectV1 = + | PublicProtocolSubjectV1 + | PublicRouteSubjectV1 + | PublicTaskSubjectV1; + +export interface PublicAssertionSummaryV1 { + id: string; + required: boolean; + passed: boolean; +} + +export interface PublicIncidentRefV1 { + corpusId: string; +} + +export interface PublicEvidenceRecordV1 { + recordId: string; + subjectId: string; + evidenceLayer: EvidenceLayer; + suiteId: string; + suiteVersion: string; + scenarioId: string; + scenarioVersion: string; + verdict: CompatibilityVerdict; + observedDayUtc: string; + subject: PublicEvidenceSubjectV1; + assertions: PublicAssertionSummaryV1[]; + incidentRefs?: PublicIncidentRefV1[]; + artifactRefs?: string[]; +} + +export interface PublicArtifactV1 { + artifactId: string; + artifactClass: string; + mediaType: string; + byteCount: number; + contentBase64: string; +} + +export interface PublicPublisherV1 { + algorithm: "ed25519"; + keyId: string; + publicKey: string; +} + +export interface PublicBundleSignatureV1 { + algorithm: "ed25519"; + signedDigest: string; + signature: string; +} + +export interface PublicEvidenceBundleUnsignedV1 { + schemaVersion: typeof PUBLIC_EVIDENCE_BUNDLE_SCHEMA_VERSION; + exportPolicyVersion: typeof PUBLIC_EXPORT_POLICY_VERSION; + bundleId: string; + createdDayUtc: string; + publisher: PublicPublisherV1; + records: PublicEvidenceRecordV1[]; + artifacts: PublicArtifactV1[]; + bundleDigest: string; +} + +export interface PublicEvidenceBundleV1 extends PublicEvidenceBundleUnsignedV1 { + signature: PublicBundleSignatureV1; +} + +export interface PublicEvidencePreviewBundleV1 { + schemaVersion: typeof PUBLIC_EVIDENCE_BUNDLE_SCHEMA_VERSION; + exportPolicyVersion: typeof PUBLIC_EXPORT_POLICY_VERSION; + createdDayUtc: string; + records: PublicEvidenceRecordV1[]; + artifacts: PublicArtifactV1[]; +} + +export type PublicRevocationReasonV1 = + | "publisher_retracted" + | "privacy_retraction" + | "evidence_invalidated" + | "superseded"; + +export interface PublicRevocationTargetV1 { + kind: "bundle" | "record"; + id: string; +} + +export interface PublicEvidenceRevocationV1 { + schemaVersion: typeof PUBLIC_EVIDENCE_REVOCATION_SCHEMA_VERSION; + revocationId: string; + issuedDayUtc: string; + publisher: PublicPublisherV1; + targets: PublicRevocationTargetV1[]; + reason: PublicRevocationReasonV1; + signature: PublicBundleSignatureV1; +} + +export type PublicRevocationVerificationResult = + | { status: "cryptographically_valid"; revocation: PublicEvidenceRevocationV1 } + | { status: "schema_rejected" | "digest_invalid" | "signature_invalid" | "publisher_mismatch" | "unknown_target"; detail?: string }; + +export interface CommunityEvidenceSummaryV1 { + trustClass: "community_untrusted_v1"; + status: "cryptographically_valid"; + bundleId: string; + publisherKeyId: string; + activeRecordCount: number; + revokedRecordCount: number; +} + +export type PublicProjectionNotExportableReason = + | "private_route_identity" + | "unsupported_subject" + | "unsafe_public_field" + | "unsupported_adapter_family"; + +export type PublicEvidenceProjectionResult = + | { status: "exportable"; record: PublicEvidenceRecordV1 } + | { status: "not_exportable"; reason: PublicProjectionNotExportableReason; detailCode?: string }; diff --git a/src/lab/public/validate.ts b/src/lab/public/validate.ts new file mode 100644 index 0000000000..ba562636a2 --- /dev/null +++ b/src/lab/public/validate.ts @@ -0,0 +1,391 @@ +import { EVIDENCE_LAYERS, VERDICTS, type EvidenceLayer } from "../constants"; +import { isSha256Hex } from "../digest"; +import { publicEvidenceId } from "./ids"; +import { findPublicRouteRegistryEntry } from "./registry"; +import { + PUBLIC_ADAPTER_FAMILIES, + PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION, + type PublicAdapterFamily, + type PublicAssertionSummaryV1, + type PublicEvidenceRecordV1, + type PublicEvidenceSubjectV1, + type PublicIncidentRefV1, + type PublicProtocolSubjectV1, + type PublicRouteRegistryEntryV1, + type PublicRouteRegistryManifestV1, + type PublicRouteSubjectV1, + type PublicTaskSubjectV1, +} from "./types"; + +const MAX_PUBLIC_STRING_BYTES = 4 * 1024; +const MAX_PUBLIC_ASSERTIONS = 64; +const MAX_PUBLIC_INCIDENT_REFS = 32; +const MAX_PUBLIC_ARTIFACT_REFS = 16; +const PUBLIC_IDENTIFIER = /^[A-Za-z0-9][A-Za-z0-9._:+-]{0,255}$/; +const UTC_DAY = /^\d{4}-\d{2}-\d{2}$/; +const SOURCE_COMMIT = /^[0-9a-f]{40}$/; + +const PUBLIC_INCIDENT_CORPUS_IDS = new Set( + Array.from({ length: 21 }, (_, index) => `IC-${String(index + 1).padStart(3, "0")}`), +); + +export class PublicEvidenceValidationError extends Error { + override readonly name = "PublicEvidenceValidationError"; + + constructor(readonly code: string, message: string) { + super(message); + } +} + +function isPlainObject(value: unknown): value is Record<string, unknown> { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +function assertObject(value: unknown, field: string): Record<string, unknown> { + if (!isPlainObject(value)) { + throw new PublicEvidenceValidationError("invalid_type", `${field} must be an object`); + } + return value; +} + +function assertKnownKeys( + raw: Record<string, unknown>, + field: string, + allowed: readonly string[], +): void { + const allow = new Set(allowed); + for (const key of Object.keys(raw)) { + if (!allow.has(key)) { + throw new PublicEvidenceValidationError("unknown_field", `${field}.${key} is not public schema`); + } + } +} + +function assertString(value: unknown, field: string, max = MAX_PUBLIC_STRING_BYTES): string { + if (typeof value !== "string") { + throw new PublicEvidenceValidationError("invalid_type", `${field} must be a string`); + } + if (value.includes("\0")) { + throw new PublicEvidenceValidationError("unsafe_public_field", `${field} contains NUL`); + } + if (new TextEncoder().encode(value).byteLength > max) { + throw new PublicEvidenceValidationError("field_too_large", `${field} exceeds ${max} bytes`); + } + return value; +} + +function assertPublicIdentifier(value: unknown, field: string): string { + const result = assertString(value, field, 256); + if (!PUBLIC_IDENTIFIER.test(result)) { + throw new PublicEvidenceValidationError("unsafe_public_field", `${field} is not a closed public identifier`); + } + return result; +} + +function assertBoolean(value: unknown, field: string): boolean { + if (value !== true && value !== false) { + throw new PublicEvidenceValidationError("invalid_type", `${field} must be boolean`); + } + return value; +} + +function assertSha256(value: unknown, field: string): string { + const result = assertString(value, field, 64); + if (!isSha256Hex(result)) { + throw new PublicEvidenceValidationError("invalid_digest", `${field} must be lowercase sha256 hex`); + } + return result; +} + +function assertClosed<T extends string>( + value: unknown, + field: string, + allowed: readonly T[], +): T { + if (typeof value !== "string" || !(allowed as readonly string[]).includes(value)) { + throw new PublicEvidenceValidationError("closed_set", `${field} is not in the public closed set`); + } + return value as T; +} + +function assertUtcDay(value: unknown, field: string): string { + const result = assertString(value, field, 10); + if (!UTC_DAY.test(result)) { + throw new PublicEvidenceValidationError("invalid_day", `${field} must be YYYY-MM-DD`); + } + const parsed = new Date(`${result}T00:00:00.000Z`); + if (Number.isNaN(parsed.getTime()) || parsed.toISOString().slice(0, 10) !== result) { + throw new PublicEvidenceValidationError("invalid_day", `${field} must be a real UTC day`); + } + return result; +} + +function validateAdapterFamily(value: unknown, field: string): PublicAdapterFamily { + return assertClosed(value, field, PUBLIC_ADAPTER_FAMILIES); +} + +function validateProtocolSubject(rawValue: unknown): PublicProtocolSubjectV1 { + const raw = assertObject(rawValue, "subject"); + assertKnownKeys(raw, "subject", [ + "subjectKind", + "compatibilityVersion", + "adapterFamily", + "inboundProtocol", + "upstreamProtocol", + "surface", + ]); + if (raw.subjectKind !== "protocol") { + throw new PublicEvidenceValidationError("layer_subject_mismatch", "protocol layer requires protocol subject"); + } + return { + subjectKind: "protocol", + compatibilityVersion: assertPublicIdentifier(raw.compatibilityVersion, "subject.compatibilityVersion"), + adapterFamily: validateAdapterFamily(raw.adapterFamily, "subject.adapterFamily"), + inboundProtocol: assertPublicIdentifier(raw.inboundProtocol, "subject.inboundProtocol"), + upstreamProtocol: assertPublicIdentifier(raw.upstreamProtocol, "subject.upstreamProtocol"), + surface: assertPublicIdentifier(raw.surface, "subject.surface"), + }; +} + +function validateRouteSubject(rawValue: unknown): PublicRouteSubjectV1 { + const raw = assertObject(rawValue, "subject"); + assertKnownKeys(raw, "subject", [ + "subjectKind", + "providerId", + "modelId", + "adapterFamily", + "compatibilityVersion", + ]); + if (raw.subjectKind !== "route") { + throw new PublicEvidenceValidationError("layer_subject_mismatch", "route layer requires route subject"); + } + const providerId = assertPublicIdentifier(raw.providerId, "subject.providerId"); + const modelId = assertPublicIdentifier(raw.modelId, "subject.modelId"); + const adapterFamily = validateAdapterFamily(raw.adapterFamily, "subject.adapterFamily"); + const entry = findPublicRouteRegistryEntry(providerId, modelId); + if (!entry || !entry.adapterFamilies.includes(adapterFamily)) { + throw new PublicEvidenceValidationError("public_registry_rejected", "route is not in the reviewed public registry"); + } + return { + subjectKind: "route", + providerId, + modelId, + adapterFamily, + compatibilityVersion: assertPublicIdentifier(raw.compatibilityVersion, "subject.compatibilityVersion"), + }; +} + +function validateTaskSubject(rawValue: unknown): PublicTaskSubjectV1 { + const raw = assertObject(rawValue, "subject"); + assertKnownKeys(raw, "subject", [ + "subjectKind", + "route", + "taskClassId", + "taskClassVersion", + "taskFixtureDigest", + "verifierManifestDigest", + "fabricCompatibilityVersion", + ]); + if (raw.subjectKind !== "task") { + throw new PublicEvidenceValidationError("layer_subject_mismatch", "task layer requires task subject"); + } + return { + subjectKind: "task", + route: validateRouteSubject(raw.route), + taskClassId: assertPublicIdentifier(raw.taskClassId, "subject.taskClassId"), + taskClassVersion: assertPublicIdentifier(raw.taskClassVersion, "subject.taskClassVersion"), + taskFixtureDigest: assertSha256(raw.taskFixtureDigest, "subject.taskFixtureDigest"), + verifierManifestDigest: assertSha256(raw.verifierManifestDigest, "subject.verifierManifestDigest"), + fabricCompatibilityVersion: assertPublicIdentifier( + raw.fabricCompatibilityVersion, + "subject.fabricCompatibilityVersion", + ), + }; +} + +function validateSubject(raw: unknown, layer: EvidenceLayer): PublicEvidenceSubjectV1 { + if (layer === "protocol_conformance") return validateProtocolSubject(raw); + if (layer === "live_route_compatibility") return validateRouteSubject(raw); + if (layer === "task_effectiveness") return validateTaskSubject(raw); + const _exhaustive: never = layer; + throw new PublicEvidenceValidationError("unsupported_layer", String(_exhaustive)); +} + +function validateAssertion(rawValue: unknown, index: number): PublicAssertionSummaryV1 { + const raw = assertObject(rawValue, `assertions[${index}]`); + assertKnownKeys(raw, `assertions[${index}]`, ["id", "required", "passed"]); + return { + id: assertPublicIdentifier(raw.id, `assertions[${index}].id`), + required: assertBoolean(raw.required, `assertions[${index}].required`), + passed: assertBoolean(raw.passed, `assertions[${index}].passed`), + }; +} + +export function isPublicIncidentRef(value: unknown): value is string { + return typeof value === "string" && PUBLIC_INCIDENT_CORPUS_IDS.has(value); +} + +function validateIncidentRef(rawValue: unknown, index: number): PublicIncidentRefV1 { + const raw = assertObject(rawValue, `incidentRefs[${index}]`); + assertKnownKeys(raw, `incidentRefs[${index}]`, ["corpusId"]); + const corpusId = assertString(raw.corpusId, `incidentRefs[${index}].corpusId`, 6); + if (!isPublicIncidentRef(corpusId)) { + throw new PublicEvidenceValidationError("incident_ref_rejected", `${corpusId} is not in the reviewed corpus`); + } + return { corpusId }; +} + +function validateUniqueIds(rawValue: unknown, field: string, max: number): string[] { + if (!Array.isArray(rawValue)) { + throw new PublicEvidenceValidationError("invalid_type", `${field} must be an array`); + } + if (rawValue.length > max) { + throw new PublicEvidenceValidationError("array_too_large", `${field} exceeds ${max}`); + } + const values = rawValue.map((value, index) => assertSha256(value, `${field}[${index}]`)); + if (new Set(values).size !== values.length) { + throw new PublicEvidenceValidationError("duplicate_id", `${field} contains duplicates`); + } + return values; +} + +export function validatePublicEvidenceRecord(rawValue: unknown): PublicEvidenceRecordV1 { + const raw = assertObject(rawValue, "record"); + assertKnownKeys(raw, "record", [ + "recordId", + "subjectId", + "evidenceLayer", + "suiteId", + "suiteVersion", + "scenarioId", + "scenarioVersion", + "verdict", + "observedDayUtc", + "subject", + "assertions", + "incidentRefs", + "artifactRefs", + ]); + + const evidenceLayer = assertClosed(raw.evidenceLayer, "record.evidenceLayer", EVIDENCE_LAYERS); + const subject = validateSubject(raw.subject, evidenceLayer); + const subjectId = assertSha256(raw.subjectId, "record.subjectId"); + const expectedSubjectId = publicEvidenceId("subject", subject); + if (subjectId !== expectedSubjectId) { + throw new PublicEvidenceValidationError("subject_id_mismatch", "record.subjectId does not match public subject"); + } + + if (!Array.isArray(raw.assertions)) { + throw new PublicEvidenceValidationError("invalid_type", "record.assertions must be an array"); + } + if (raw.assertions.length > MAX_PUBLIC_ASSERTIONS) { + throw new PublicEvidenceValidationError("array_too_large", `record.assertions exceeds ${MAX_PUBLIC_ASSERTIONS}`); + } + const assertions = raw.assertions.map(validateAssertion); + + let incidentRefs: PublicIncidentRefV1[] | undefined; + if (raw.incidentRefs !== undefined) { + if (!Array.isArray(raw.incidentRefs)) { + throw new PublicEvidenceValidationError("invalid_type", "record.incidentRefs must be an array"); + } + if (raw.incidentRefs.length > MAX_PUBLIC_INCIDENT_REFS) { + throw new PublicEvidenceValidationError( + "array_too_large", + `record.incidentRefs exceeds ${MAX_PUBLIC_INCIDENT_REFS}`, + ); + } + incidentRefs = raw.incidentRefs.map(validateIncidentRef); + const ids = incidentRefs.map((ref) => ref.corpusId); + if (new Set(ids).size !== ids.length) { + throw new PublicEvidenceValidationError("duplicate_id", "record.incidentRefs contains duplicates"); + } + } + + const artifactRefs = raw.artifactRefs === undefined + ? undefined + : validateUniqueIds(raw.artifactRefs, "record.artifactRefs", MAX_PUBLIC_ARTIFACT_REFS); + + const withoutRecordId: Omit<PublicEvidenceRecordV1, "recordId"> = { + subjectId, + evidenceLayer, + suiteId: assertPublicIdentifier(raw.suiteId, "record.suiteId"), + suiteVersion: assertPublicIdentifier(raw.suiteVersion, "record.suiteVersion"), + scenarioId: assertPublicIdentifier(raw.scenarioId, "record.scenarioId"), + scenarioVersion: assertPublicIdentifier(raw.scenarioVersion, "record.scenarioVersion"), + verdict: assertClosed(raw.verdict, "record.verdict", VERDICTS), + observedDayUtc: assertUtcDay(raw.observedDayUtc, "record.observedDayUtc"), + subject, + assertions, + ...(incidentRefs !== undefined ? { incidentRefs } : {}), + ...(artifactRefs !== undefined ? { artifactRefs } : {}), + }; + const recordId = assertSha256(raw.recordId, "record.recordId"); + const expectedRecordId = publicEvidenceId("record", withoutRecordId); + if (recordId !== expectedRecordId) { + throw new PublicEvidenceValidationError("record_id_mismatch", "record.recordId does not match public record"); + } + return { recordId, ...withoutRecordId }; +} + +function validateRegistryEntry(rawValue: unknown, index: number): PublicRouteRegistryEntryV1 { + const raw = assertObject(rawValue, `entries[${index}]`); + assertKnownKeys(raw, `entries[${index}]`, ["providerId", "modelId", "adapterFamilies"]); + if (!Array.isArray(raw.adapterFamilies) || raw.adapterFamilies.length === 0) { + throw new PublicEvidenceValidationError("invalid_registry", `entries[${index}].adapterFamilies must be non-empty`); + } + const adapterFamilies = raw.adapterFamilies.map((value, adapterIndex) => + validateAdapterFamily(value, `entries[${index}].adapterFamilies[${adapterIndex}]`) + ); + if (new Set(adapterFamilies).size !== adapterFamilies.length) { + throw new PublicEvidenceValidationError("duplicate_id", `entries[${index}].adapterFamilies contains duplicates`); + } + return { + providerId: assertPublicIdentifier(raw.providerId, `entries[${index}].providerId`), + modelId: assertPublicIdentifier(raw.modelId, `entries[${index}].modelId`), + adapterFamilies, + }; +} + +export function validatePublicRouteRegistryManifest(rawValue: unknown): PublicRouteRegistryManifestV1 { + const raw = assertObject(rawValue, "publicRouteRegistry"); + assertKnownKeys(raw, "publicRouteRegistry", [ + "schemaVersion", + "registryVersion", + "sourceCommit", + "entries", + "manifestDigest", + ]); + if (raw.schemaVersion !== PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION) { + throw new PublicEvidenceValidationError("unsupported_version", "unsupported public route registry schema"); + } + const registryVersion = assertPublicIdentifier(raw.registryVersion, "publicRouteRegistry.registryVersion"); + const sourceCommit = assertString(raw.sourceCommit, "publicRouteRegistry.sourceCommit", 40); + if (!SOURCE_COMMIT.test(sourceCommit)) { + throw new PublicEvidenceValidationError("invalid_registry", "publicRouteRegistry.sourceCommit must be a commit SHA"); + } + if (!Array.isArray(raw.entries) || raw.entries.length === 0 || raw.entries.length > 512) { + throw new PublicEvidenceValidationError("invalid_registry", "publicRouteRegistry.entries must contain 1..512 entries"); + } + const entries = raw.entries.map(validateRegistryEntry); + const identities = entries.map((entry) => `${entry.providerId}\0${entry.modelId}`); + if (new Set(identities).size !== identities.length) { + throw new PublicEvidenceValidationError("duplicate_id", "publicRouteRegistry.entries contains duplicates"); + } + const manifestDigest = assertSha256(raw.manifestDigest, "publicRouteRegistry.manifestDigest"); + const expectedDigest = publicEvidenceId("route_registry", { + schemaVersion: PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION, + registryVersion, + sourceCommit, + entries, + }); + if (manifestDigest !== expectedDigest) { + throw new PublicEvidenceValidationError("digest_invalid", "publicRouteRegistry.manifestDigest mismatch"); + } + return { + schemaVersion: PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION, + registryVersion, + sourceCommit, + entries, + manifestDigest, + }; +} diff --git a/src/lib/lab-activation.ts b/src/lib/lab-activation.ts index dc37f85af1..18298b7fe2 100644 --- a/src/lib/lab-activation.ts +++ b/src/lib/lab-activation.ts @@ -29,6 +29,7 @@ import { labAutomationPolicyPath } from "../lab/paths"; import type { OcxConfig } from "../types"; import { LabAutomationError } from "../lab/automation/types"; import { registerLabPassiveRouteLinker } from "./lab-passive-linker-registration"; +import { registerCurrentServerResourceCleanup } from "./server-resource-ownership"; import { setCompatibilityEvidenceProvider } from "../routing/compatibility/provider-slot"; import { labCompatibilityEvidenceProvider } from "../routing/compatibility/lab-evidence-provider"; import { @@ -37,8 +38,18 @@ import { } from "../lab/automation/orchestrator"; import { createProductionLabRouteExecutor } from "./lab-live-route-production"; +interface LabRuntimeBinding { + release(): void; +} + +interface LabActivationRecord { + staticDetach: Array<() => void>; + runtime: LabRuntimeBinding | null; + seenRuntimeConfigs: WeakSet<OcxConfig>; +} + /** Activation records keyed by configDir, so one process can own several configs. */ -const activated = new Map<string, Array<() => void>>(); +const activated = new Map<string, LabActivationRecord>(); const activationKey = (configDir?: string): string => configDir ?? ""; @@ -83,60 +94,110 @@ export function labActivationRequired(config: OcxConfig, configDir?: string): bo return labAutomationEnabledOnDisk(configDir); } +function startAutomationIfEnabled(configDir?: string): void { + if (!labAutomationEnabledOnDisk(configDir)) return; + try { + startLabAutomationScheduler(configDir); + } catch (err) { + // Neither a malformed automation file nor a busy state lock may take the proxy down + // at startup. Lab automation stays off for this run; routing, evidence, and every + // other subsystem keep working. + // + // The two causes get different messages because they need different actions, and a + // lock-contention failure reported as "invalid config" sends the operator to fix a + // file that is fine. Contention can also stall startup by up to the 5s lock wait. + const code = err instanceof LabAutomationError ? err.code : null; + if (code === "state_lock_busy" || code === "state_lock_failed") { + console.warn( + "[lab] Lab automation did not start: another process holds the automation state lock." + + " Automation stays off for this run and will be retried on the next start.", + ); + } else { + console.warn( + "[lab] Lab automation is disabled for this run because its configuration could not be" + + " loaded:", + err instanceof Error ? err.message : err, + ); + } + } +} + +function installLabAutomationRuntime( + record: LabActivationRecord, + config: OcxConfig, + configDir?: string, +): void { + const previous = record.runtime; + const routeExecutor = createProductionLabRouteExecutor({ configDir, loadConfig: () => config }); + const releaseDispatchDeps = setLabAutomationDispatchDeps({ + configDir, + loadConfig: () => config, + routeExecutor, + }); + + let released = false; + let detachOwnerCleanup = () => {}; + const binding: LabRuntimeBinding = { + release() { + if (released) return; + released = true; + detachOwnerCleanup(); + releaseDispatchDeps(); + if (record.runtime === binding) record.runtime = null; + }, + }; + // `setLabAutomationDispatchDeps` already registers its own owner cleanup. This second + // receipt only keeps the activation record in sync with that owner-scoped lifetime, so a + // later same-process server can see that the static Lab slots survived but CL-08 authority + // did not. The release is idempotent, so cleanup order does not matter. + detachOwnerCleanup = registerCurrentServerResourceCleanup(binding.release); + + // Install the successor before releasing the predecessor. The dispatcher token check then + // makes the predecessor release a no-op for the successor scheduler/authority. + record.runtime = binding; + record.seenRuntimeConfigs.add(config); + previous?.release(); +} + /** - * Register Lab into the core slots. Idempotent per configDir and safe to call again after - * a routing profile is created at runtime. + * Register Lab into the core slots. Static activation is idempotent per configDir. The + * server-owned CL-08 runtime binding is refreshed when its prior owner ended or when a new + * server instance arrives with a config object that has not owned this activation before. */ export function activateLab(config: OcxConfig, configDir?: string): void { const key = activationKey(configDir); - // INVARIANT: activation is all-or-nothing and reason-independent. Every slot is - // registered here regardless of WHY activation was required, which is what makes this - // key safe as configDir alone -- an automation-only activation still installs the - // compatibility provider a later profile needs. If any registration ever becomes - // conditional on the activation reason, this key must include that reason, or the early - // return will silently skip it forever. - if (activated.has(key)) return; - - const detach: Array<() => void> = []; - detach.push(registerLabPassiveRouteLinker(configDir)); - detach.push(setCompatibilityEvidenceProvider(labCompatibilityEvidenceProvider)); + const existing = activated.get(key); + if (existing) { + // A released predecessor leaves the static Lab slots resident but removes dispatch + // authority and its scheduler. A live successor uses a fresh config object. Rebind in + // either case, but never let an older already-seen server steal authority back from a + // newer successor merely because it receives another management request. + if (existing.runtime === null || !existing.seenRuntimeConfigs.has(config)) { + installLabAutomationRuntime(existing, config, configDir); + startAutomationIfEnabled(configDir); + } + return; + } - const routeExecutor = createProductionLabRouteExecutor({ configDir, loadConfig: () => config }); - detach.push(setLabAutomationDispatchDeps({ configDir, loadConfig: () => config, routeExecutor })); + // INVARIANT: static activation is all-or-nothing and reason-independent. Every static + // slot is registered here regardless of WHY activation was required, so automation-only + // activation still installs the compatibility provider a later profile needs. + const record: LabActivationRecord = { + staticDetach: [], + runtime: null, + seenRuntimeConfigs: new WeakSet<OcxConfig>(), + }; + record.staticDetach.push(registerLabPassiveRouteLinker(configDir)); + record.staticDetach.push(setCompatibilityEvidenceProvider(labCompatibilityEvidenceProvider)); + installLabAutomationRuntime(record, config, configDir); // Record the activation BEFORE the scheduler start. startLabAutomationScheduler runs the // full automation normalizer, which throws on any field violation, and this call sits on // the startup path of every install that has a routing profile. Storing the record first // means a throw cannot orphan the detach receipts and leave slots registered with no - // activation record -- which would let a later activateLab register them a second time. - activated.set(key, detach); - - if (labAutomationEnabledOnDisk(configDir)) { - try { - startLabAutomationScheduler(configDir); - } catch (err) { - // Neither a malformed automation file nor a busy state lock may take the proxy down - // at startup. Lab automation stays off for this run; routing, evidence, and every - // other subsystem keep working. - // - // The two causes get different messages because they need different actions, and a - // lock-contention failure reported as "invalid config" sends the operator to fix a - // file that is fine. Contention can also stall startup by up to the 5s lock wait. - const code = err instanceof LabAutomationError ? err.code : null; - if (code === "state_lock_busy" || code === "state_lock_failed") { - console.warn( - "[lab] Lab automation did not start: another process holds the automation state lock." - + " Automation stays off for this run and will be retried on the next start.", - ); - } else { - console.warn( - "[lab] Lab automation is disabled for this run because its configuration could not be" - + " loaded:", - err instanceof Error ? err.message : err, - ); - } - } - } + // activation record, which would let a later activateLab register them a second time. + activated.set(key, record); + startAutomationIfEnabled(configDir); } /** True when this configDir has been activated. */ @@ -152,9 +213,10 @@ export function isLabActivated(configDir?: string): boolean { * users who never opted in. */ export function resetLabActivationForTests(): void { - for (const [key, detach] of [...activated]) { + for (const [key, record] of [...activated]) { activated.delete(key); - for (const release of [...detach].reverse()) { + try { record.runtime?.release(); } catch { /* teardown is best-effort */ } + for (const release of [...record.staticDetach].reverse()) { try { release(); } catch { /* teardown is best-effort */ } } } diff --git a/src/lib/lab-live-pinned-sender.ts b/src/lib/lab-live-pinned-sender.ts index d25966d3af..0c336511bd 100644 --- a/src/lib/lab-live-pinned-sender.ts +++ b/src/lib/lab-live-pinned-sender.ts @@ -19,22 +19,33 @@ export function createLabAuthorizedPinnedSender( headers, maxBytes: limits.maxOutputBytes, connectTimeoutMs: limits.connectTimeoutMs, - idleTimeoutMs: Math.min(limits.firstByteTimeoutMs, limits.inactivityTimeoutMs), + firstByteTimeoutMs: limits.firstByteTimeoutMs, + inactivityTimeoutMs: limits.inactivityTimeoutMs, rejectUnauthorized: true, context: "Lab provider response", }; let response: Response; + let body: string; try { response = request.method === "POST" ? await pinnedHttpPost(url, pinned, request.body ?? "", signal, options) : await pinnedHttpGet(url, pinned, signal, options); + body = await response.text(); } catch (error) { - if (error instanceof PinnedHttpError && error.code === "connect_timeout") { - throw new TransportError("connect_timeout", "pinned provider connection timed out"); + if (error instanceof PinnedHttpError) { + switch (error.code) { + case "connect_timeout": + throw new TransportError("connect_timeout", "pinned provider connection timed out"); + case "first_byte_timeout": + throw new TransportError("first_byte_timeout", "pinned provider first byte timed out"); + case "inactivity_timeout": + throw new TransportError("inactivity_timeout", "pinned provider response stalled"); + case "output_byte_limit": + throw new TransportError("output_byte_limit", "pinned provider response exceeded byte budget"); + } } throw error; } - const body = await response.text(); const responseHeaders: Record<string, string> = {}; for (const headerName of LAB_RESPONSE_HEADER_ALLOWLIST) { const value = response.headers.get(headerName); @@ -42,4 +53,4 @@ export function createLabAuthorizedPinnedSender( } return { status: response.status, headers: responseHeaders, body }; }; -} \ No newline at end of file +} diff --git a/src/lib/pinned-http.ts b/src/lib/pinned-http.ts index 247c945818..97e8d91a9c 100644 --- a/src/lib/pinned-http.ts +++ b/src/lib/pinned-http.ts @@ -3,7 +3,11 @@ import https from "node:https"; export type PinnedAddress = { address: string; family: number }; -export type PinnedHttpErrorCode = "connect_timeout"; +export type PinnedHttpErrorCode = + | "connect_timeout" + | "first_byte_timeout" + | "inactivity_timeout" + | "output_byte_limit"; export class PinnedHttpError extends Error { override readonly name = "PinnedHttpError"; @@ -15,6 +19,11 @@ export interface PinnedHttpRequestOptions { maxBytes?: number; /** Optional deadline for establishing the TCP connection and, for HTTPS, completing TLS. */ connectTimeoutMs?: number; + /** Optional deadline from connection establishment until response headers arrive. */ + firstByteTimeoutMs?: number; + /** Optional maximum idle interval between response-body chunks. */ + inactivityTimeoutMs?: number; + /** @deprecated Use firstByteTimeoutMs and inactivityTimeoutMs. */ idleTimeoutMs?: number; rejectUnauthorized?: boolean; context?: string; @@ -37,7 +46,11 @@ function pinnedHttpRequest( } const context = options?.context ?? "request"; const connectTimeoutMs = options?.connectTimeoutMs; - const idleTimeoutMs = options?.idleTimeoutMs ?? 60_000; + const legacyIdleTimeoutMs = options?.idleTimeoutMs ?? 60_000; + const usesLegacyIdleTimeout = options?.firstByteTimeoutMs === undefined + && options?.inactivityTimeoutMs === undefined; + const firstByteTimeoutMs = options?.firstByteTimeoutMs ?? legacyIdleTimeoutMs; + const inactivityTimeoutMs = options?.inactivityTimeoutMs ?? legacyIdleTimeoutMs; const maxBytes = options?.maxBytes; const headers = new Headers(options?.headers); headers.set("host", parsed.host); @@ -56,17 +69,30 @@ function pinnedHttpRequest( let settled = false; let req: ClientRequest | undefined; let connectTimer: ReturnType<typeof setTimeout> | undefined; + let firstByteTimer: ReturnType<typeof setTimeout> | undefined; const clearConnectTimer = () => { if (connectTimer !== undefined) clearTimeout(connectTimer); connectTimer = undefined; }; + const clearFirstByteTimer = () => { + if (firstByteTimer !== undefined) clearTimeout(firstByteTimer); + firstByteTimer = undefined; + }; const fail = (error: unknown) => { clearConnectTimer(); + clearFirstByteTimer(); try { req?.destroy(); } catch { /* ignore */ } if (settled) return; settled = true; reject(error instanceof Error ? error : new Error(String(error))); }; + const startFirstByteTimer = () => { + clearFirstByteTimer(); + firstByteTimer = setTimeout( + () => fail(new PinnedHttpError("first_byte_timeout", `${context} first byte timed out`)), + firstByteTimeoutMs, + ); + }; const requestOptions: RequestOptions & { servername?: string } = { protocol: parsed.protocol, hostname: parsed.hostname, @@ -101,6 +127,7 @@ function pinnedHttpRequest( const onResponse = (response: IncomingMessage) => { clearConnectTimer(); + clearFirstByteTimer(); const status = response.statusCode ?? 0; const responseHeaders = new Headers(); for (const [key, value] of Object.entries(response.headers)) { @@ -124,28 +151,35 @@ function pinnedHttpRequest( let received = 0; const stream = new ReadableStream<Uint8Array>({ start(controller) { - response.setTimeout(idleTimeoutMs, () => { - const error = new Error(`${context} stalled`); - fail(error); + let bodySettled = false; + const failBody = (error: Error) => { + if (bodySettled) return; + bodySettled = true; try { controller.error(error); } catch { /* closed */ } + try { response.destroy(); } catch { /* ignore */ } + try { req?.destroy(); } catch { /* ignore */ } + }; + + response.setTimeout(inactivityTimeoutMs, () => { + failBody(new PinnedHttpError("inactivity_timeout", `${context} stalled`)); }); response.on("data", (chunk: Buffer | string) => { + if (bodySettled) return; const buffer = typeof chunk === "string" ? Buffer.from(chunk) : chunk; received += buffer.byteLength; if (maxBytes !== undefined && received > maxBytes) { - const error = new Error(`${context} exceeds ${maxBytes} byte cap`); - fail(error); - try { controller.error(error); } catch { /* closed */ } + failBody(new PinnedHttpError("output_byte_limit", `${context} exceeds ${maxBytes} byte cap`)); return; } try { controller.enqueue(buffer); } catch { /* closed */ } }); response.on("end", () => { + if (bodySettled) return; + bodySettled = true; try { controller.close(); } catch { /* closed */ } }); response.on("error", (error: Error) => { - fail(error); - try { controller.error(error); } catch { /* closed */ } + failBody(error); }); }, cancel() { @@ -163,20 +197,40 @@ function pinnedHttpRequest( const onAbort = () => fail(signal?.reason instanceof Error ? signal.reason : new Error("aborted")); signal?.addEventListener("abort", onAbort, { once: true }); req.on("socket", (socket) => { - if (!socket.connecting || connectTimeoutMs === undefined) return; const connectedEvent = parsed.protocol === "https:" ? "secureConnect" : "connect"; - connectTimer = setTimeout(() => fail(new PinnedHttpError("connect_timeout", `${context} connect timed out`)), connectTimeoutMs); - socket.once(connectedEvent, clearConnectTimer); - socket.once("error", clearConnectTimer); - socket.once("close", clearConnectTimer); + if (!socket.connecting) { + if (!usesLegacyIdleTimeout) startFirstByteTimer(); + return; + } + if (connectTimeoutMs !== undefined) { + connectTimer = setTimeout( + () => fail(new PinnedHttpError("connect_timeout", `${context} connect timed out`)), + connectTimeoutMs, + ); + } + socket.once(connectedEvent, () => { + clearConnectTimer(); + if (!usesLegacyIdleTimeout) startFirstByteTimer(); + }); + socket.once("error", () => { + clearConnectTimer(); + clearFirstByteTimer(); + }); + socket.once("close", () => { + clearConnectTimer(); + clearFirstByteTimer(); + }); }); - req.setTimeout(idleTimeoutMs, () => fail(new Error(`${context} timed out`))); + if (usesLegacyIdleTimeout) { + req.setTimeout(legacyIdleTimeoutMs, () => fail(new Error(`${context} timed out`))); + } req.on("error", error => { signal?.removeEventListener("abort", onAbort); fail(error); }); req.on("close", () => { clearConnectTimer(); + clearFirstByteTimer(); signal?.removeEventListener("abort", onAbort); }); req.end(body); diff --git a/src/providers/derive.ts b/src/providers/derive.ts index 0aef212180..e43ed866b9 100644 --- a/src/providers/derive.ts +++ b/src/providers/derive.ts @@ -1,10 +1,14 @@ import type { CodexAccountMode, OcxProviderConfig } from "../types"; import { PROVIDER_REGISTRY, - providerMatchesRegistryTransport, registryEntryForProviderDestination, type ProviderRegistryEntry, } from "./registry"; +import { + providerMatchesRegistryTransportWithStaticGuards, + registryEntrySupportsLiveModelDiscovery, + repairStaticModelCatalogProvider, +} from "./static-model-discovery"; export interface DerivedKeyLoginProvider { label: string; @@ -204,6 +208,7 @@ export function applyDirectReasoningEffortContracts( * keep distinguishing local runtimes from API-key providers after the seed round-trip. */ export function providerConfigSeed(entry: ProviderRegistryEntry): OcxProviderConfig { + const liveModels = registryEntrySupportsLiveModelDiscovery(entry) ? entry.liveModels : false; return { adapter: entry.adapter, baseUrl: entry.baseUrl, @@ -219,7 +224,7 @@ export function providerConfigSeed(entry: ProviderRegistryEntry): OcxProviderCon ...(entry.staticHeaders ? { headers: { ...entry.staticHeaders } } : {}), ...(entry.defaultModel ? { defaultModel: entry.defaultModel } : {}), ...(entry.models ? { models: [...entry.models] } : {}), - ...(entry.liveModels !== undefined ? { liveModels: entry.liveModels } : {}), + ...(liveModels !== undefined ? { liveModels } : {}), ...(entry.contextWindow !== undefined ? { contextWindow: entry.contextWindow } : {}), ...(entry.modelContextWindows ? { modelContextWindows: { ...entry.modelContextWindows } } : {}), ...(entry.modelInputModalities ? { modelInputModalities: cloneRecordOfArrays(entry.modelInputModalities) } : {}), @@ -263,6 +268,7 @@ export function deriveKeyLoginMap(): Record<string, DerivedKeyLoginProvider> { for (const entry of PROVIDER_REGISTRY) { if (entry.authKind !== "key") continue; if (!entry.dashboardUrl) throw new Error(`Registry key provider missing dashboardUrl: ${entry.id}`); + const liveModels = registryEntrySupportsLiveModelDiscovery(entry) ? entry.liveModels : false; out[entry.id] = { label: entry.label, baseUrl: entry.baseUrl, @@ -272,7 +278,7 @@ export function deriveKeyLoginMap(): Record<string, DerivedKeyLoginProvider> { ...(entry.apiKeyTransport !== undefined ? { apiKeyTransport: entry.apiKeyTransport } : {}), dashboardUrl: entry.dashboardUrl, ...(entry.models ? { models: [...entry.models] } : {}), - ...(entry.liveModels !== undefined ? { liveModels: entry.liveModels } : {}), + ...(liveModels !== undefined ? { liveModels } : {}), ...(entry.defaultModel ? { defaultModel: entry.defaultModel } : {}), ...(entry.contextWindow !== undefined ? { contextWindow: entry.contextWindow } : {}), ...(entry.modelContextWindows ? { modelContextWindows: { ...entry.modelContextWindows } } : {}), @@ -378,7 +384,7 @@ function enrichReasoningSummariesByDestination(prov: OcxProviderConfig): void { export function enrichProviderFromRegistry(name: string, prov: OcxProviderConfig): void { const entry = PROVIDER_REGISTRY.find(row => row.id === name); - if (!entry || !providerMatchesRegistryTransport(name, prov)) { + if (!entry || !providerMatchesRegistryTransportWithStaticGuards(name, prov)) { // Name lookup failed, but the row may still point at a vendor route we know. #1100 was // reported against a hand-added provider literally named "GLM": routing worked, yet every // piece of registry metadata was skipped because no registry id is called "GLM". @@ -395,6 +401,7 @@ export function enrichProviderFromRegistry(name: string, prov: OcxProviderConfig modelReasoningEffortMap: prov.modelReasoningEffortMap, }; const seed = providerConfigSeed(entry); + repairStaticModelCatalogProvider(name, prov); if (prov.apiKeyTransport === undefined && seed.apiKeyTransport !== undefined) prov.apiKeyTransport = seed.apiKeyTransport; if (!prov.defaultModel && seed.defaultModel) prov.defaultModel = seed.defaultModel; if (prov.responsesPath === undefined && seed.responsesPath !== undefined) prov.responsesPath = seed.responsesPath; diff --git a/src/providers/registry.ts b/src/providers/registry.ts index 1bc02fd900..60cce440fe 100644 --- a/src/providers/registry.ts +++ b/src/providers/registry.ts @@ -903,6 +903,7 @@ const CLINE_PASS_MODELS = [ "cline-pass/mimo-v2.5", "cline-pass/mimo-v2.5-pro", "cline-pass/minimax-m3", + "cline-pass/qwen3.8-max", "cline-pass/qwen3.7-max", "cline-pass/qwen3.7-plus", ]; @@ -928,9 +929,10 @@ const CLINE_PASS_IMAGE_MODELS = new Set([ "cline-pass/minimax-m3", "cline-pass/qwen3.7-plus", ]); -const CLINE_PASS_TEXT_ONLY_MODELS = CLINE_PASS_MODELS.filter(id => !CLINE_PASS_IMAGE_MODELS.has(id)); +const CLINE_PASS_MODALITY_KNOWN_MODELS = CLINE_PASS_MODELS.filter(id => id !== "cline-pass/qwen3.8-max"); +const CLINE_PASS_TEXT_ONLY_MODELS = CLINE_PASS_MODALITY_KNOWN_MODELS.filter(id => !CLINE_PASS_IMAGE_MODELS.has(id)); const CLINE_PASS_MODEL_INPUT_MODALITIES: Record<string, string[]> = Object.fromEntries( - CLINE_PASS_MODELS.map(id => [id, CLINE_PASS_IMAGE_MODELS.has(id) ? ["text", "image"] : ["text"]]), + CLINE_PASS_MODALITY_KNOWN_MODELS.map(id => [id, CLINE_PASS_IMAGE_MODELS.has(id) ? ["text", "image"] : ["text"]]), ); export const PROVIDER_REGISTRY: readonly ProviderRegistryEntry[] = [ diff --git a/src/providers/static-model-discovery.ts b/src/providers/static-model-discovery.ts new file mode 100644 index 0000000000..c1dc568f18 --- /dev/null +++ b/src/providers/static-model-discovery.ts @@ -0,0 +1,86 @@ +import type { OcxProviderConfig } from "../types"; +import { + getProviderRegistryEntry, + providerMatchesRegistryTransport, + type ProviderRegistryEntry, +} from "./registry"; + +const STATIC_MODEL_CATALOG_PROVIDER_IDS = new Set(["cline-pass", "mimo-free"]); + +function normalizedEndpoint(value: string): string { + const trimmed = value.trim(); + try { + const parsed = new URL(trimmed); + parsed.pathname = parsed.pathname.replace(/\/+$/, "") || "/"; + return parsed.toString().replace(/\/$/, ""); + } catch { + return trimmed.replace(/\/+$/, ""); + } +} + +function exactRegistryTransportMatch( + entry: ProviderRegistryEntry, + provider: Pick<OcxProviderConfig, "baseUrl" | "adapter"> & Partial<Pick<OcxProviderConfig, "authMode">>, + options: { allowLegacyMimoLocal?: boolean } = {}, +): boolean { + if (entry.allowBaseUrlOverride || /\{[^}]*\}/.test(entry.baseUrl)) return false; + if (typeof provider.baseUrl !== "string" || provider.adapter !== entry.adapter) return false; + const legacyMimoLocal = options.allowLegacyMimoLocal === true + && entry.id === "mimo-free" + && provider.authMode === "local"; + if (provider.authMode !== undefined && provider.authMode !== "key" && !legacyMimoLocal) return false; + return normalizedEndpoint(provider.baseUrl) === normalizedEndpoint(entry.baseUrl); +} + +/** Registry policy for providers whose maintained model list is authoritative. */ +export function registryEntrySupportsLiveModelDiscovery(entry: ProviderRegistryEntry): boolean { + return !STATIC_MODEL_CATALOG_PROVIDER_IDS.has(entry.id); +} + +/** + * Static-catalog authority is tied to canonical provider identity and exact transport. + * Renamed/custom rows stay operator-owned: Cline and ClinePass intentionally share a transport, + * so destination matching alone cannot safely identify a renamed ClinePass configuration. + */ +export function staticModelCatalogEntryForProvider( + name: string, + provider: OcxProviderConfig, +): ProviderRegistryEntry | undefined { + const entry = getProviderRegistryEntry(name); + if (!entry || !STATIC_MODEL_CATALOG_PROVIDER_IDS.has(entry.id)) return undefined; + return exactRegistryTransportMatch(entry, provider, { allowLegacyMimoLocal: true }) + ? entry + : undefined; +} + +export function providerSupportsLiveModelDiscovery(name: string, provider: OcxProviderConfig): boolean { + return staticModelCatalogEntryForProvider(name, provider) === undefined; +} + +/** + * MiMo Free predates collision preservation in the registry. Keep same-named custom rows out of + * registry ownership without broadening the generic transport matcher to other key providers. + */ +export function providerMatchesRegistryTransportWithStaticGuards( + name: string, + provider: Pick<OcxProviderConfig, "baseUrl" | "adapter"> & Partial<Pick<OcxProviderConfig, "authMode">>, +): boolean { + if (name !== "mimo-free") return providerMatchesRegistryTransport(name, provider); + const entry = getProviderRegistryEntry(name); + return entry !== undefined + && exactRegistryTransportMatch(entry, provider, { allowLegacyMimoLocal: true }); +} + +/** Repair only registry-owned legacy state; operator-owned model lists stay untouched. */ +export function repairStaticModelCatalogProvider(name: string, provider: OcxProviderConfig): void { + const entry = staticModelCatalogEntryForProvider(name, provider); + if (!entry) return; + provider.liveModels = false; + if ( + name === "mimo-free" + && entry.id === "mimo-free" + && (provider.authMode === undefined || provider.authMode === "local") + ) { + provider.authMode = "key"; + } +} diff --git a/src/reasoning-effort.ts b/src/reasoning-effort.ts index 5e97994e11..ff168f4b7d 100644 --- a/src/reasoning-effort.ts +++ b/src/reasoning-effort.ts @@ -19,6 +19,30 @@ export function isCodexReasoningEffort(effort: string): boolean { return CODEX_REASONING_SET.has(effort); } +/** + * True for ladder members plus the `none`/`minimal` sentinels. Both are valid declared + * efforts (OpenAI accepts `minimal`; Codex validates `none` against + * `supported_reasoning_levels` for no-reasoning subagent spawns, #883/#962) but are NOT + * part of the low..ultra ladder: they never appear in default ladders, ranks, or clamps + * (`minimal` is mapped to `low` on the wire by requestToCodexEffort). + */ +export function isDeclaredReasoningEffort(effort: string): boolean { + return effort === "none" || effort === "minimal" || CODEX_REASONING_SET.has(effort); +} + +/** + * Reorder any declared subset (low..ultra, plus the optional `none`/`minimal` sentinels + * first, in that order) into canonical order and drop duplicates. Catalog + * `supported_reasoning_levels` follow the input order and the fallback default picks the + * first entry, so a caller-chosen order would otherwise leak into the catalog. + */ +export function canonicalizeReasoningEfforts(values: readonly string[]): string[] { + const seen = new Set(values); + const ordered = CODEX_REASONING_ORDER.filter(effort => seen.has(effort)); + const sentinels = ["none", "minimal"].filter(effort => seen.has(effort)); + return [...sentinels, ...ordered]; +} + /** * Reasoning ladder accepted for the OpenAI vision sidecar. `ultra` is deliberately excluded: * the vision describer is a single helper call, and `ultra` would be collapsed to `max` by the @@ -66,7 +90,9 @@ export function sanitizeCodexReasoningEfforts(efforts: readonly string[] | undef const seen = new Set<string>(); const out: string[] = []; for (const effort of efforts) { - if (!CODEX_REASONING_SET.has(effort) || seen.has(effort)) continue; + // `none`/`minimal` are valid declared sentinels, kept and sorted first (rank -1); they + // never appear in the default ladder. + if ((effort !== "none" && effort !== "minimal" && !CODEX_REASONING_SET.has(effort)) || seen.has(effort)) continue; seen.add(effort); out.push(effort); } diff --git a/src/router.ts b/src/router.ts index 12edcd532f..6d258f2944 100644 --- a/src/router.ts +++ b/src/router.ts @@ -11,8 +11,12 @@ import type { NormalizedComboConfig } from "./combos/types"; import { hasOwnProvider, resolveEnvValue } from "./config"; import { assertProviderDestinationAllowed } from "./lib/destination-policy"; import { redactSecretString, redactUrlForLog } from "./lib/redact"; -import { PROVIDER_REGISTRY, providerCodexAccountMode, providerMatchesRegistryTransport } from "./providers/registry"; +import { PROVIDER_REGISTRY, providerCodexAccountMode } from "./providers/registry"; import { applyDirectReasoningEffortContracts } from "./providers/derive"; +import { + providerMatchesRegistryTransportWithStaticGuards, + providerSupportsLiveModelDiscovery, +} from "./providers/static-model-discovery"; import { isCanonicalOpenAiForwardProvider, LEGACY_CHATGPT_PROVIDER_ID, @@ -86,7 +90,7 @@ const MODEL_PROVIDER_PATTERNS: Array<{ providerNames: string[]; prefixes: string export function knownModelIdsForProvider(provName: string, prov: OcxProviderConfig): string[] { const ids = new Set<string>(); for (const id of prov.models ?? []) ids.add(id); - const registry = providerMatchesRegistryTransport(provName, prov) + const registry = providerMatchesRegistryTransportWithStaticGuards(provName, prov) ? PROVIDER_REGISTRY.find(entry => entry.id === provName) : undefined; for (const id of registry?.models ?? []) ids.add(id); @@ -249,20 +253,26 @@ function usableResolvedApiKey(apiKey: string | undefined): string | undefined { export function routedProviderConfig(providerName: string, provider: OcxProviderConfig): OcxProviderConfig { const registryEntry = PROVIDER_REGISTRY.find(entry => entry.id === providerName); - if (!registryEntry || !providerMatchesRegistryTransport(providerName, provider)) { + if (!registryEntry || !providerMatchesRegistryTransportWithStaticGuards(providerName, provider)) { assertProviderDestinationAllowed(providerName, provider); return { ...provider, apiKey: usableResolvedApiKey(provider.apiKey) }; } const resolvedApiKey = usableResolvedApiKey(provider.apiKey); + const staticModelCatalog = !providerSupportsLiveModelDiscovery(providerName, provider); + const repairLegacyMimoFreeAuth = providerName === "mimo-free" + && staticModelCatalog + && (provider.authMode === undefined || provider.authMode === "local"); const explicitKeyOverride = registryEntry.authKind === "oauth" && registryEntry.allowKeyAuthOverride === true && provider.authMode === "key" && resolvedApiKey !== undefined; const canonicalAuthMode = explicitKeyOverride ? "key" - : registryEntry.authKind === "forward" || registryEntry.authKind === "oauth" - ? registryEntry.authKind - : provider.authMode === "forward" ? undefined : provider.authMode; + : repairLegacyMimoFreeAuth + ? "key" + : registryEntry.authKind === "forward" || registryEntry.authKind === "oauth" + ? registryEntry.authKind + : provider.authMode === "forward" ? undefined : provider.authMode; const reasoningEffortMap = mergeRecord(registryEntry.reasoningEffortMap, provider.reasoningEffortMap); const modelReasoningEffortMap = mergeNestedRecord(registryEntry.modelReasoningEffortMap, provider.modelReasoningEffortMap); const modelReasoningEfforts = mergeStringArrayRecord(registryEntry.modelReasoningEfforts, provider.modelReasoningEfforts); @@ -343,6 +353,7 @@ export function routedProviderConfig(providerName: string, provider: OcxProvider : {}), authMode: canonicalAuthMode, apiKey: resolvedApiKey, + ...(staticModelCatalog ? { liveModels: false } : {}), // Backfill the Google wire mode + Vertex project/location from the registry when the user // config omits them, so a minimal `google-vertex`/`google-antigravity` entry still routes // through the correct branch (CCA/Vertex) instead of falling back to AI Studio. diff --git a/src/server/adapter-resolve.ts b/src/server/adapter-resolve.ts index 2edf7e3ee9..8587b3191d 100644 --- a/src/server/adapter-resolve.ts +++ b/src/server/adapter-resolve.ts @@ -1,12 +1,4 @@ -import { createAnthropicAdapter } from "../adapters/anthropic"; -import { createAzureAdapter } from "../adapters/azure"; -import { createCursorAdapter } from "../adapters/cursor"; -import { createGoogleAdapter } from "../adapters/google"; -import { createKiroAdapter } from "../adapters/kiro"; -import { createMimoFreeAdapter } from "../adapters/mimo-free"; -import { createOpenAIChatAdapter } from "../adapters/openai-chat"; -import { createCommandCodeAdapter } from "../adapters/command-code"; -import { createResponsesPassthroughAdapter } from "../adapters/openai-responses"; +import { createRegisteredAdapter } from "../adapters/registry"; import type { OcxProviderConfig } from "../types"; import { isWirePinnedModel, MODEL_ADAPTER_OVERRIDE_ALLOWED, pinnedWireAdapter } from "../types"; import { isCanonicalOpenAiForwardProvider } from "../providers/openai-tiers"; @@ -57,27 +49,5 @@ export function resolveWireProtocolOverride( /** Build the provider adapter for a resolved provider config. */ export function resolveAdapter(providerConfig: OcxProviderConfig, cacheRetention?: "none" | "short" | "long") { - switch (providerConfig.adapter) { - case "command-code": - return createCommandCodeAdapter(providerConfig); - case "openai-chat": - return createOpenAIChatAdapter(providerConfig); - case "anthropic": - return createAnthropicAdapter(providerConfig, cacheRetention); - case "openai-responses": - return createResponsesPassthroughAdapter(providerConfig); - case "google": - return createGoogleAdapter(providerConfig); - case "kiro": - return createKiroAdapter(providerConfig); - case "azure": - case "azure-openai": - return createAzureAdapter(providerConfig); - case "cursor": - return createCursorAdapter(providerConfig); - case "mimo-free": - return createMimoFreeAdapter(providerConfig); - default: - throw new Error(`Unknown adapter: ${providerConfig.adapter}`); - } + return createRegisteredAdapter(providerConfig, { cacheRetention }); } diff --git a/src/server/management/config-routes.ts b/src/server/management/config-routes.ts index 0306a02f4b..2e171587be 100644 --- a/src/server/management/config-routes.ts +++ b/src/server/management/config-routes.ts @@ -134,6 +134,7 @@ function publicVisionSidecarSettings( export async function handleConfigRoutes(ctx: ManagementContext): Promise<Response | null> { const { req, url, config, deps, convergeCodexCatalog, syncClaudeAgentDefsBestEffort } = ctx; + const readStartupHealth = deps.getCachedStartupHealth ?? getCachedStartupHealth; if (url.pathname === "/api/config" && req.method === "GET") { return jsonResponse(safeConfigDTO(config)); } @@ -189,7 +190,7 @@ export async function handleConfigRoutes(ctx: ManagementContext): Promise<Respon streamMode: config.streamMode ?? "auto", appOwnedMemoryBudgetMb: config.appOwnedMemoryBudgetMb ?? 256, codexAccountPickerEnabled: codexAccountPickerEnabled(config), - startupHealth: await getCachedStartupHealth(config), + startupHealth: await readStartupHealth(config), codexRuntime: { path: displayCodexRuntimePath(resolved.runtime.command), version: resolved.runtime.version, @@ -211,7 +212,7 @@ export async function handleConfigRoutes(ctx: ManagementContext): Promise<Respon } if (url.pathname === "/api/startup-health" && req.method === "GET") { - return jsonResponse(await getCachedStartupHealth(config)); + return jsonResponse(await readStartupHealth(config)); } if (url.pathname === "/api/startup-action" && req.method === "POST") { @@ -368,7 +369,7 @@ export async function handleConfigRoutes(ctx: ManagementContext): Promise<Respon appOwnedMemoryBudgetMb: config.appOwnedMemoryBudgetMb ?? 256, codexAccountPickerEnabled: pickerIsEnabled, catalogRefreshPending, - startupHealth: await getCachedStartupHealth(config), + startupHealth: await readStartupHealth(config), }); } diff --git a/src/server/management/context.ts b/src/server/management/context.ts index ecfaabb1d3..d049df2dc2 100644 --- a/src/server/management/context.ts +++ b/src/server/management/context.ts @@ -1,5 +1,6 @@ import type { OcxConfig } from "../../types"; import type { NativeProfileApiDeps } from "../../codex/native-profile-api"; +import type { StartupHealth } from "../../codex/autostart-health"; import type { StartupInstallAction } from "../startup-action-control"; import type { ManagementPrincipal } from "../management-auth"; import type { CatalogModel } from "../../codex/catalog"; @@ -16,6 +17,8 @@ export interface ManagementApiDeps { toggleCodexMultiAgentV2?: (enabled: boolean) => void; toggleDefaultModeRequestUserInput?: (enabled: boolean) => void; createManagementConvergeCodex?: (config: Readonly<OcxConfig>) => ConvergeCodex; + /** Startup-health seam keeps route tests from launching platform probes. */ + getCachedStartupHealth?: (config: Pick<OcxConfig, "codexAutoStart">) => Promise<StartupHealth>; /** * Persistence seam for route-level tests. Production leaves this unset and uses * `saveConfigPreservingClaudeCode`; tests that pass an in-memory fixture config diff --git a/src/server/management/lab-routes.ts b/src/server/management/lab-routes.ts index 5c0929e47f..9370c246c7 100644 --- a/src/server/management/lab-routes.ts +++ b/src/server/management/lab-routes.ts @@ -14,6 +14,7 @@ */ import { + ARTIFACT_CLASSES, EVIDENCE_LAYERS, EXECUTION_MODES, EVENT_KINDS, @@ -30,6 +31,7 @@ import { LabProjectionIncompatibleError, LabProjectionUnavailableError, LAB_QUERY_MAX_PAGE_SIZE, + PASSIVE_PRODUCTION_MAX_LIMIT, queryLabArtifactByDigest, queryLabArtifacts, queryLabCatalogEntries, @@ -42,6 +44,15 @@ import { queryLabVerdicts, queryPassiveProductionSignals, } from "../../lab/query"; +import { + exportLocalPublicEvidence, + importCommunityEvidenceValue, + listCommunityEvidenceContext, + parseStrictPublicJson, + previewLocalPublicEvidence, + summarizePublicEvidenceVerification, + PublicEvidenceValidationError, +} from "../../lab/public"; import { jsonResponse } from "../auth-cors"; import type { ManagementContext } from "./context"; @@ -75,20 +86,24 @@ function projectionErrorResponse(err: unknown, ctx: ManagementContext): Response return null; } -function parseLimit(raw: string | null, ctx: ManagementContext): number | undefined | Response { +function parseLimit( + raw: string | null, + ctx: ManagementContext, + max = LAB_QUERY_MAX_PAGE_SIZE, +): number | undefined | Response { const parsed = raw === null ? undefined : parseQueryInt(raw); if (parsed === "invalid") { return errorResponse( "invalid_limit", - `limit must be an integer from 1 to ${LAB_QUERY_MAX_PAGE_SIZE}`, + `limit must be an integer from 1 to ${max}`, 400, ctx, ); } - if (parsed !== undefined && (parsed < 1 || parsed > LAB_QUERY_MAX_PAGE_SIZE)) { + if (parsed !== undefined && (parsed < 1 || parsed > max)) { return errorResponse( "invalid_limit", - `limit must be an integer from 1 to ${LAB_QUERY_MAX_PAGE_SIZE}`, + `limit must be an integer from 1 to ${max}`, 400, ctx, ); @@ -155,7 +170,7 @@ function parseExecutionMode(raw: string | null, ctx: ManagementContext): Executi if (!raw) return undefined; const trimmed = raw.trim(); if (!EXECUTION_MODES.includes(trimmed as ExecutionMode)) { - return errorResponse("invalid_execution_mode", "executionMode must be a supported execution mode", 400, ctx); + return errorResponse("invalid_execution_mode", "executionMode must be a supported lab execution mode", 400, ctx); } return trimmed as ExecutionMode; } @@ -186,9 +201,154 @@ function paginatedEnvelope<T>(page: { items: T[]; nextCursor?: string; hasMore: }; } +const MAX_PUBLIC_REQUEST_BYTES = 2 * 1024 * 1024; + +async function readBoundedPublicJson(req: Request): Promise<unknown> { + const lengthRaw = req.headers.get("content-length"); + if (lengthRaw) { + const length = Number(lengthRaw); + if (!Number.isFinite(length) || length < 0 || length > MAX_PUBLIC_REQUEST_BYTES) { + throw new PublicEvidenceValidationError( + "public_request_too_large", + "public evidence request exceeds 2 MiB", + ); + } + } + if (!req.body) { + throw new PublicEvidenceValidationError("public_request_body", "JSON body is required"); + } + const reader = req.body.getReader(); + const chunks: Uint8Array[] = []; + let total = 0; + while (true) { + const { done, value } = await reader.read(); + if (done) break; + total += value.byteLength; + if (total > MAX_PUBLIC_REQUEST_BYTES) { + await reader.cancel(); + throw new PublicEvidenceValidationError( + "public_request_too_large", + "public evidence request exceeds 2 MiB", + ); + } + chunks.push(value); + } + const bytes = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + bytes.set(chunk, offset); + offset += chunk.byteLength; + } + return parseStrictPublicJson(bytes, "public evidence request"); +} + +function publicEventIds(raw: unknown): string[] { + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + throw new PublicEvidenceValidationError("public_request_body", "request body must be an object"); + } + const keys = Object.keys(raw); + if (keys.length !== 1 || keys[0] !== "eventIds") { + throw new PublicEvidenceValidationError("public_request_body", "only eventIds is accepted"); + } + const eventIds = (raw as { eventIds?: unknown }).eventIds; + if (!Array.isArray(eventIds) || !eventIds.every((value) => typeof value === "string")) { + throw new PublicEvidenceValidationError("public_request_body", "eventIds must be a string array"); + } + return eventIds as string[]; +} + +function publicBundleValue(raw: unknown): unknown { + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + throw new PublicEvidenceValidationError("public_request_body", "request body must be an object"); + } + const keys = Object.keys(raw); + if (keys.length !== 1 || keys[0] !== "bundle") { + throw new PublicEvidenceValidationError("public_request_body", "only bundle is accepted"); + } + return (raw as { bundle?: unknown }).bundle; +} + +function publicErrorResponse(err: unknown, ctx: ManagementContext): Response { + if (err instanceof PublicEvidenceValidationError) { + const status = err.code === "community_cache_busy" ? 503 : 400; + const response = errorResponse(err.code, err.message, status, ctx); + if (status === 503) response.headers.set("Retry-After", "1"); + return response; + } + const projected = projectionErrorResponse(err, ctx); + if (projected) return projected; + return errorResponse("public_evidence_internal", "internal public evidence failure", 500, ctx); +} + export async function handleLabRoutes(ctx: ManagementContext): Promise<Response | null> { const { url, req, config } = ctx; if (!url.pathname.startsWith("/api/lab")) return null; + + if (req.method === "GET" && url.pathname === "/api/lab/public/community") { + try { + return jsonResponse(listCommunityEvidenceContext(), 200, req, config); + } catch (err) { + return publicErrorResponse(err, ctx); + } + } + + if (req.method === "POST") { + if (url.pathname === "/api/lab/public/preview") { + try { + const body = await readBoundedPublicJson(req); + return jsonResponse( + previewLocalPublicEvidence({ eventIds: publicEventIds(body) }), + 200, + req, + config, + ); + } catch (err) { + return publicErrorResponse(err, ctx); + } + } + if (url.pathname === "/api/lab/public/export") { + try { + const body = await readBoundedPublicJson(req); + return jsonResponse( + exportLocalPublicEvidence({ eventIds: publicEventIds(body) }), + 200, + req, + config, + ); + } catch (err) { + return publicErrorResponse(err, ctx); + } + } + if (url.pathname === "/api/lab/public/verify") { + try { + const body = await readBoundedPublicJson(req); + const result = summarizePublicEvidenceVerification(publicBundleValue(body)); + return jsonResponse( + result, + result.status === "cryptographically_valid" ? 200 : 400, + req, + config, + ); + } catch (err) { + return publicErrorResponse(err, ctx); + } + } + if (url.pathname === "/api/lab/public/community/import") { + try { + const body = await readBoundedPublicJson(req); + return jsonResponse( + importCommunityEvidenceValue(publicBundleValue(body)), + 200, + req, + config, + ); + } catch (err) { + return publicErrorResponse(err, ctx); + } + } + return null; + } + if (req.method !== "GET") return null; if (url.pathname === "/api/lab/status") { @@ -198,7 +358,7 @@ export async function handleLabRoutes(ctx: ManagementContext): Promise<Response if (url.pathname === "/api/lab/production-signals") { const subjectId = url.searchParams.get("subjectId")?.trim(); if (!subjectId) return errorResponse("invalid_subject", "subjectId is required", 400, ctx); - const limit = parseLimit(url.searchParams.get("limit"), ctx); + const limit = parseLimit(url.searchParams.get("limit"), ctx, PASSIVE_PRODUCTION_MAX_LIMIT); if (limit instanceof Response) return limit; try { return jsonResponse(queryPassiveProductionSignals(subjectId, limit), 200, req, config); @@ -213,10 +373,7 @@ export async function handleLabRoutes(ctx: ManagementContext): Promise<Response if (layerParsed instanceof Response) return layerParsed; const suiteId = url.searchParams.get("suiteId")?.trim() || url.searchParams.get("suite")?.trim() || undefined; try { - const scenarios = queryLabCatalogEntries({ - layer: layerParsed, - suiteId, - }); + const scenarios = queryLabCatalogEntries({ layer: layerParsed, suiteId }); return jsonResponse({ scenarios }, 200, req, config); } catch (err) { const mapped = projectionErrorResponse(err, ctx); @@ -255,11 +412,7 @@ export async function handleLabRoutes(ctx: ManagementContext): Promise<Response const limit = parseLimit(url.searchParams.get("limit"), ctx); if (limit instanceof Response) return limit; try { - const page = queryLabSubjects( - url.searchParams.get("kind")?.trim() || undefined, - url.searchParams.get("cursor"), - limit, - ); + const page = queryLabSubjects(url.searchParams.get("kind")?.trim() || undefined, url.searchParams.get("cursor"), limit); return jsonResponse(paginatedEnvelope(page, "subjects"), 200, req, config); } catch (err) { const mapped = projectionErrorResponse(err, ctx); @@ -323,9 +476,10 @@ export async function handleLabRoutes(ctx: ManagementContext): Promise<Response const eventKind = parseEventKind(url.searchParams.get("eventKind"), ctx); if (eventKind instanceof Response) return eventKind; const excludedRaw = url.searchParams.get("excluded"); - let excluded: boolean | undefined; - if (excludedRaw === "true") excluded = true; - else if (excludedRaw === "false") excluded = false; + if (excludedRaw !== null && excludedRaw !== "true" && excludedRaw !== "false") { + return errorResponse("invalid_excluded", "excluded must be true or false", 400, ctx); + } + const excluded = excludedRaw === "true" ? true : excludedRaw === "false" ? false : undefined; try { const page = queryLabEvents({ eventKind, @@ -367,6 +521,14 @@ export async function handleLabRoutes(ctx: ManagementContext): Promise<Response if (statusRaw && !["present", "corrupt", "purged_unavailable"].includes(statusRaw)) { return errorResponse("invalid_status", "status must be present, corrupt, or purged_unavailable", 400, ctx); } + if (artifactClass && !ARTIFACT_CLASSES.includes(artifactClass as (typeof ARTIFACT_CLASSES)[number])) { + return errorResponse( + "invalid_artifact_class", + "artifactClass must be a supported artifact class", + 400, + ctx, + ); + } try { const page = queryLabArtifacts({ status: statusRaw as "present" | "corrupt" | "purged_unavailable" | undefined, @@ -398,4 +560,4 @@ export async function handleLabRoutes(ctx: ManagementContext): Promise<Response } return null; -} +} \ No newline at end of file diff --git a/src/server/management/model-routes.ts b/src/server/management/model-routes.ts index 1016029953..0cf22043f4 100644 --- a/src/server/management/model-routes.ts +++ b/src/server/management/model-routes.ts @@ -29,6 +29,46 @@ function readInputModalities(raw: unknown): { values?: string[]; error?: string } return { values: raw as string[] }; } + +/** + * Custom-row reasoning ladder. Labels are validated against the Codex ladder (low..ultra) + * exactly like provider `modelReasoningEfforts` values; unknown labels would otherwise + * surface in a catalog the upstream never accepts. An empty array is meaningful (explicit + * "no reasoning" hides the effort control) and must be preserved, not cleared. + */ +function readReasoningEfforts(raw: unknown): { values?: string[]; error?: string } { + if (raw === undefined) return {}; + if (!Array.isArray(raw)) return { error: "reasoningEfforts must be an array" }; + const rejected: string[] = []; + const values: string[] = []; + for (const value of raw) { + if (typeof value !== "string") return { error: "reasoningEfforts must contain only strings" }; + if (!isDeclaredReasoningEffort(value)) { rejected.push(value); continue; } + if (!values.includes(value)) values.push(value); + } + if (rejected.length > 0) { + return { error: `unsupported reasoning effort: ${rejected.join(", ")} (allowed: none, minimal, low, medium, high, xhigh, max, ultra)` }; + } + // Canonical order: the catalog writes supported_reasoning_levels in input order and the + // fallback default picks the first entry, so a caller-chosen order must not leak through. + return { values: canonicalizeReasoningEfforts(values) }; +} + +/** Default effort must be a ladder member that the declared ladder actually includes. */ +function readDefaultReasoningEffort(raw: unknown, efforts: string[] | undefined): { value?: string; error?: string } { + if (raw === undefined) return {}; + if (raw === null) return { value: undefined }; + if (typeof raw !== "string" || !isDeclaredReasoningEffort(raw)) { + return { error: "defaultReasoningEffort must be one of: none, minimal, low, medium, high, xhigh, max, ultra" }; + } + if (efforts === undefined || efforts.length === 0) { + return { error: "defaultReasoningEffort requires a non-empty reasoningEfforts ladder" }; + } + if (!efforts.includes(raw)) { + return { error: `defaultReasoningEffort "${raw}" is not in the declared reasoningEfforts ladder` }; + } + return { value: raw }; +} import type { CatalogModel } from "../../codex/catalog"; import { accountBoundNativeOpenAiSlugsBySelector, catalogModelSlug, configuredNativeAliasSlugs, disabledNativeSlugs, invalidateCodexModelsCache, nativeModelRows, shouldIncludeAccountBoundNativeOpenAi, uniqueCatalogModelsForPublicList } from "../../codex/catalog"; import { CatalogGatherBusyError } from "../../codex/catalog/provider-fetch"; @@ -71,6 +111,7 @@ import { parseRange, parseUsageSurface, summarizeUsage } from "../../usage/summa import { stripCodexRuntimeProviderFields } from "../../codex/auth-context"; import { getProviderRegistryEntry } from "../../providers/registry"; import { getDebugLogEntries } from "../../lib/debug-log-buffer"; +import { canonicalizeReasoningEfforts, isDeclaredReasoningEffort } from "../../reasoning-effort"; import { getInjectionDebugLogEntries } from "../../lib/injection-debug-log"; import { clearDebugSettings, @@ -330,7 +371,7 @@ export async function handleModelRoutes(ctx: ManagementContext): Promise<Respons } if (url.pathname === "/api/custom-models" && req.method === "POST") { - let body: { provider?: unknown; modelId?: unknown; displayName?: unknown; contextWindow?: unknown; inputModalities?: unknown }; + let body: { provider?: unknown; modelId?: unknown; displayName?: unknown; contextWindow?: unknown; inputModalities?: unknown; reasoningEfforts?: unknown; defaultReasoningEffort?: unknown }; try { body = await readManagementJsonBody(req); } catch (error) { rethrowManagementBodyTooLarge(error); return jsonResponse({ error: "invalid JSON body" }, 400); } const provider = typeof body.provider === "string" ? body.provider.trim() : ""; const modelId = typeof body.modelId === "string" ? body.modelId.trim() : ""; @@ -344,6 +385,10 @@ export async function handleModelRoutes(ctx: ManagementContext): Promise<Respons const modalities = readInputModalities(body.inputModalities); if (modalities.error) return jsonResponse({ error: modalities.error }, 400); const inputModalities = modalities.values; + const reasoning = readReasoningEfforts(body.reasoningEfforts); + if (reasoning.error) return jsonResponse({ error: reasoning.error }, 400); + const defaultEffort = readDefaultReasoningEffort(body.defaultReasoningEffort, reasoning.values); + if (defaultEffort.error) return jsonResponse({ error: defaultEffort.error }, 400); const existing = config.customModels ?? []; const newSlug = routedSlug(provider, modelId); if (existing.some(cm => routedSlug(cm.provider, cm.modelId) === newSlug)) { @@ -356,6 +401,8 @@ export async function handleModelRoutes(ctx: ManagementContext): Promise<Respons ...(displayName ? { displayName } : {}), ...(contextWindow ? { contextWindow } : {}), ...(inputModalities && inputModalities.length > 0 ? { inputModalities } : {}), + ...(reasoning.values !== undefined ? { reasoningEfforts: reasoning.values } : {}), + ...(defaultEffort.value ? { defaultReasoningEffort: defaultEffort.value } : {}), addedAt: new Date().toISOString(), }; config.customModels = [...existing, entry]; @@ -368,7 +415,7 @@ export async function handleModelRoutes(ctx: ManagementContext): Promise<Respons if (customPutMatch && req.method === "PUT") { let id: string; try { id = decodeURIComponent(customPutMatch[1]); } catch { return jsonResponse({ error: "invalid id encoding" }, 400); } - let body: { displayName?: unknown; contextWindow?: unknown; inputModalities?: unknown; modelId?: unknown }; + let body: { displayName?: unknown; contextWindow?: unknown; inputModalities?: unknown; modelId?: unknown; reasoningEfforts?: unknown; defaultReasoningEffort?: unknown }; try { body = await readManagementJsonBody(req); } catch (error) { rethrowManagementBodyTooLarge(error); return jsonResponse({ error: "invalid JSON body" }, 400); } const list = config.customModels ?? []; const idx = list.findIndex(cm => cm.id === id); @@ -391,6 +438,33 @@ export async function handleModelRoutes(ctx: ManagementContext): Promise<Respons if (edited.error) return jsonResponse({ error: edited.error }, 400); cm.inputModalities = edited.values && edited.values.length > 0 ? edited.values : undefined; } + // `null` clears the stored ladder back to "inherit from the provider row"; `[]` stays + // stored as an explicit "no reasoning" override. The default effort rides along and is + // validated against the ladder the row ends up with. + if (body.reasoningEfforts !== undefined) { + if (body.reasoningEfforts === null) { + cm.reasoningEfforts = undefined; + } else { + const edited = readReasoningEfforts(body.reasoningEfforts); + if (edited.error) return jsonResponse({ error: edited.error }, 400); + cm.reasoningEfforts = edited.values; + } + } + if (body.defaultReasoningEffort !== undefined) { + const edited = readDefaultReasoningEffort(body.defaultReasoningEffort, cm.reasoningEfforts); + if (edited.error) return jsonResponse({ error: edited.error }, 400); + cm.defaultReasoningEffort = edited.value; + } + // Mirror of the POST invariant: a default only survives as a member of the final ladder. + // Without this, a ladder shrink/clear on a row that was created with a default leaves a + // stale default that re-applies itself onto the inherited ladder in the generated catalog + // (the GUI toggle-off path sends only reasoningEfforts, never the default). + if (cm.defaultReasoningEffort !== undefined) { + const ladder = cm.reasoningEfforts; + if (!ladder || ladder.length === 0 || !ladder.includes(cm.defaultReasoningEffort)) { + cm.defaultReasoningEffort = undefined; + } + } const updatedSlug = routedSlug(cm.provider, cm.modelId); if (list.some((other, i) => i !== idx && routedSlug(other.provider, other.modelId) === updatedSlug)) { return jsonResponse({ error: "duplicate model" }, 409); diff --git a/src/server/management/model-rows.ts b/src/server/management/model-rows.ts index fa83d10adf..957aa68901 100644 --- a/src/server/management/model-rows.ts +++ b/src/server/management/model-rows.ts @@ -91,6 +91,14 @@ export async function listManagementModelRows(config: OcxConfig): Promise<Manage displayName: cm.displayName, ...(cm.contextWindow ? { contextWindow: cm.contextWindow } : {}), ...(cm.inputModalities ? { inputModalities: cm.inputModalities } : {}), + // Stored override, not the inherited ladder: the edit dialog must show what the user + // set (including an explicit empty "no reasoning" ladder), not what the provider row + // happens to advertise today. + ...(Array.isArray(cm.reasoningEfforts) ? { reasoningEfforts: [...cm.reasoningEfforts] } : {}), + // The stored default rides along so a client reloading /api/models can restore the + // full edit state; the GUI has no default-effort control today, but dropping it here + // would make any future PUT-based edit lose it silently. + ...(cm.defaultReasoningEffort ? { defaultReasoningEffort: cm.defaultReasoningEffort } : {}), }; }); const publicModels = uniqueCatalogModelsForPublicList(models); diff --git a/src/types.ts b/src/types.ts index 6939c07900..9bca99661f 100644 --- a/src/types.ts +++ b/src/types.ts @@ -587,6 +587,13 @@ export interface OcxCustomModel { contextWindow?: number; /** 입력 모달리티 (선택, 기본 ["text"]) */ inputModalities?: string[]; + /** + * Reasoning ladder (Codex labels) this custom row explicitly advertises. An empty array + * hides the effort control; an omitted key leaves the provider-derived ladder in charge. + */ + reasoningEfforts?: string[]; + /** Default effort label when `reasoningEfforts` is non-empty. */ + defaultReasoningEffort?: string; /** 추가 시각 (ISO 8601) */ addedAt?: string; } diff --git a/structure/10_adapter-registry.md b/structure/10_adapter-registry.md new file mode 100644 index 0000000000..41bbc20d6f --- /dev/null +++ b/structure/10_adapter-registry.md @@ -0,0 +1,36 @@ +# Adapter registry authority + +## Decision + +Runtime adapter construction has one authority: `src/adapters/registry.ts`. + +`src/server/adapter-resolve.ts` may resolve a provider/model onto an adapter id, but it does not maintain a second adapter factory inventory. The selected persisted/configured adapter id remains an untrusted string until the registry lookup succeeds. Unknown ids fail with the existing `Unknown adapter: <id>` error instead of widening configuration types around a closed compile-time union. + +## Semantic inheritance is not constructor inheritance + +Some adapters share another adapter's routed-tool semantics while retaining independent runtime construction: + +- `azure` and `azure-openai` inherit the `openai-responses` contract. +- `mimo-free` inherits the `openai-chat` contract. +- `cursor` stays direct because its `runTurn` transport and gated native-file fallback are distinct. + +The registry records those relationships with `contractParent`. A parent relationship does **not** mean the registry recursively constructs a parent adapter and injects it into the child. Azure and MiMo keep owning their existing internal composition. This avoids making production constructors depend on test/conformance needs and keeps this authority refactor behavior-neutral. + +## Wrapper-cycle and runtime validation policy + +`effectiveAdapterContract()` follows `contractParent` links at runtime with a visited set. Unknown parents and cycles fail closed. This is intentionally runtime validation: registry/config values can originate in persisted files written by older or hand-edited installations, so compile-time typing alone is not an adequate boundary. + +## Extension policy + +Adding a production adapter requires: + +1. one `ADAPTER_REGISTRY` entry with its factory; +2. either a direct `wire` + mutation contract or an explicit `contractParent`; +3. provider/model adapter ids that point only at registered ids; +4. registry-derived conformance coverage in the follow-up conformance layer. + +Do not add a second switch/list of adapter factories in request routing. Focused tests may construct a concrete adapter directly when they are testing that adapter itself; cross-adapter production routing should use registry authority. + +## Scope boundary + +This decision does not change routed `apply_patch` behavior, Cursor structured-edit conversion, Azure/MiMo request construction, or provider wire selection. Those behaviors remain owned by their existing modules and focused tests. The registry exposes the universe and semantic relationships; the next stack layer consumes that metadata for generic conformance. diff --git a/tests/adapter-buffered-tool-conformance.test.ts b/tests/adapter-buffered-tool-conformance.test.ts new file mode 100644 index 0000000000..6c2f11b928 --- /dev/null +++ b/tests/adapter-buffered-tool-conformance.test.ts @@ -0,0 +1,175 @@ +import { describe, expect, test } from "bun:test"; +import { adapterDefinitions, createRegisteredAdapter, effectiveAdapterContract, type AdapterWire } from "../src/adapters/registry"; +import { buildResponseJSON } from "../src/bridge"; +import { encodeMessage } from "../src/lib/eventstream-decoder"; +import { parseRequest } from "../src/responses/parser"; +import { buildToolBridgeMaps } from "../src/server/responses"; +import type { OcxProviderConfig } from "../src/types"; +import { createTestTranslatorBudget } from "./helpers/translator-budget"; + +const PATCH = `*** Begin Patch +*** Add File: buffered-안녕.txt ++quote: "double" ++slash: \\ path ++unicode: 世界 +*** End Patch`; + +const WIRE_MODELS: Record<AdapterWire, string> = { + "openai-chat": "grok-4.6", + anthropic: "claude-haiku-4-5", + google: "gemini-3.5-flash", + "command-code": "deepseek/deepseek-v4-flash", + kiro: "claude-sonnet-4.5", + "openai-responses": "deepseek-v4-flash", + cursor: "cursor/auto", +}; + +function providerFixture(adapterId: string, wire: AdapterWire): OcxProviderConfig { + const baseUrls: Record<AdapterWire, string> = { + "openai-chat": "https://api.x.ai/v1", + anthropic: "https://api.anthropic.com", + google: "https://generativelanguage.googleapis.com", + "command-code": "https://api.commandcode.ai", + kiro: "https://runtime.us-east-1.kiro.dev", + "openai-responses": "https://api.deepseek.com", + cursor: "https://api2.cursor.sh", + }; + const baseUrl = adapterId === "mimo-free" + ? "https://api.xiaomimimo.com/api/free-ai/openai/chat" + : adapterId === "azure" || adapterId === "azure-openai" + ? "https://example.openai.azure.com/openai/v1" + : baseUrls[wire]; + return { + adapter: adapterId, + baseUrl, + authMode: wire === "anthropic" || wire === "command-code" ? "oauth" : "key", + apiKey: wire === "kiro" ? "ksk_test" : "test-key", + defaultMaxOutputTokens: 64_000, + googleMode: "ai-studio", + ...(wire === "openai-responses" ? { responsesPath: "/responses" } : {}), + } as OcxProviderConfig; +} + +function parsed(wire: AdapterWire) { + const value = parseRequest({ + model: WIRE_MODELS[wire], + input: "Apply the exact patch.", + stream: false, + tools: [{ type: "custom", name: "apply_patch", description: "Apply a patch" }], + }); + if (wire === "kiro") value._kiroAuthContext = { apiRegion: "us-east-1" }; + return value; +} + +const kiroEncoder = new TextEncoder(); +function kiroFrame(payload: unknown): Uint8Array { + return encodeMessage( + { ":message-type": "event", ":event-type": "toolUseEvent" }, + kiroEncoder.encode(JSON.stringify(payload)), + ); +} + +function bufferedResponse(wire: AdapterWire, wireName = "apply_patch"): Response | undefined { + const args = { input: PATCH }; + if (wire === "openai-chat") { + return new Response(JSON.stringify({ + choices: [{ + message: { + role: "assistant", + tool_calls: [{ + id: "call_buffered_patch", + type: "function", + function: { name: wireName, arguments: JSON.stringify(args) }, + }], + }, + finish_reason: "tool_calls", + }], + usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 }, + })); + } + if (wire === "anthropic") { + return new Response(JSON.stringify({ + content: [{ type: "tool_use", id: "call_buffered_patch", name: wireName, input: args }], + stop_reason: "tool_use", + usage: { input_tokens: 1, output_tokens: 1 }, + })); + } + if (wire === "google") { + return new Response(JSON.stringify({ + candidates: [{ + content: { parts: [{ functionCall: { name: wireName, args } }] }, + finishReason: "STOP", + }], + usageMetadata: { promptTokenCount: 1, candidatesTokenCount: 1, totalTokenCount: 2 }, + })); + } + if (wire === "command-code") { + return new Response([ + JSON.stringify({ + type: "tool-call", + toolCallId: "call_buffered_patch", + toolName: wireName, + input: args, + }), + JSON.stringify({ type: "finish", rawFinishReason: "tool_use" }), + ].join("\n")); + } + if (wire === "kiro") { + const frames = [ + kiroFrame({ name: wireName, toolUseId: "call_buffered_patch" }), + kiroFrame({ input: JSON.stringify(args), name: wireName, toolUseId: "call_buffered_patch" }), + kiroFrame({ name: wireName, stop: true, toolUseId: "call_buffered_patch" }), + ]; + let index = 0; + return new Response(new ReadableStream<Uint8Array>({ + pull(controller) { + if (index < frames.length) controller.enqueue(frames[index++]!); + else controller.close(); + }, + })); + } + return undefined; +} + +function restoredInput(output: unknown): string | undefined { + if (!Array.isArray(output)) return undefined; + const call = output.find(item => + item && typeof item === "object" + && (item as Record<string, unknown>).type === "custom_tool_call" + && (item as Record<string, unknown>).name === "apply_patch" + ) as Record<string, unknown> | undefined; + return typeof call?.input === "string" ? call.input : undefined; +} + +describe("registry-derived buffered tool conformance", () => { + test("every buffered parser restores hostile freeform input exactly", async () => { + let covered = 0; + for (const [adapterId] of adapterDefinitions()) { + const contract = effectiveAdapterContract(adapterId); + const adapter = createRegisteredAdapter(providerFixture(adapterId, contract.wire)); + if (!adapter.parseResponse) continue; + if (contract.wire === "openai-responses") { + // Responses passthrough only invokes parseResponse for routed compaction, where tool calls + // are not part of the contract. Azure inherits that same compaction-only parser. + expect(["openai-responses", "azure", "azure-openai"]).toContain(adapterId); + continue; + } + const response = bufferedResponse(contract.wire); + expect(response, `${adapterId}:${contract.wire}`).toBeDefined(); + if (!response) continue; + covered += 1; + + const request = parsed(contract.wire); + const events = await adapter.parseResponse(response, createTestTranslatorBudget()); + const maps = buildToolBridgeMaps(request); + const built = buildResponseJSON(events, request.modelId, { + toolNsMap: maps.toolNsMap, + declaredToolNames: maps.declaredToolNames, + freeformToolNames: maps.freeformToolNames, + toolSearchToolNames: maps.toolSearchToolNames, + }); + expect(restoredInput(built.output), adapterId).toBe(PATCH); + } + expect(covered).toBeGreaterThan(0); + }); +}); diff --git a/tests/adapter-registry-authority.test.ts b/tests/adapter-registry-authority.test.ts new file mode 100644 index 0000000000..fa10f27984 --- /dev/null +++ b/tests/adapter-registry-authority.test.ts @@ -0,0 +1,102 @@ +import { describe, expect, test } from "bun:test"; +import { + adapterDefinitions, + createRegisteredAdapter, + effectiveAdapterContract, + getAdapterDefinition, +} from "../src/adapters/registry"; +import { resolveAdapter } from "../src/server/adapter-resolve"; +import type { OcxParsedRequest, OcxProviderConfig } from "../src/types"; +import { withTestTranslatorBudget } from "./helpers/translator-budget"; + +const EXPECTED_ADAPTER_NAMES = { + "command-code": "command-code", + "openai-chat": "openai-chat", + anthropic: "anthropic", + "openai-responses": "openai-responses", + google: "google", + kiro: "kiro", + azure: "azure-openai", + "azure-openai": "azure-openai", + cursor: "cursor", + "mimo-free": "mimo-free", +} as const; + +function provider(adapter: string): OcxProviderConfig { + return { + adapter, + // mimo-free throws for non-canonical endpoints since #1714; every other + // adapter accepts the placeholder URL. + baseUrl: adapter === "mimo-free" + ? "https://api.xiaomimimo.com/api/free-ai/openai/chat" + : "https://example.invalid/v1", + authMode: "key", + apiKey: "test-key", + defaultMaxOutputTokens: 4096, + } as OcxProviderConfig; +} + +const ANTHROPIC_CACHE_REQUEST: OcxParsedRequest = { + modelId: "claude-haiku-4-5", + stream: true, + options: {}, + context: { + messages: [{ role: "user", content: "cache me", timestamp: 0 }], + }, +}; + +async function expectLongCacheRetention(adapter: ReturnType<typeof resolveAdapter>): Promise<void> { + const request = await withTestTranslatorBudget(adapter).buildRequest(ANTHROPIC_CACHE_REQUEST); + const body = JSON.parse(request.body) as { + messages?: Array<{ content?: string | Array<{ cache_control?: { type?: string; ttl?: string } }> }>; + }; + const content = body.messages?.[0]?.content; + if (!Array.isArray(content)) throw new Error("expected Anthropic cache retention to annotate user content"); + expect(content.at(-1)?.cache_control).toEqual({ type: "ephemeral", ttl: "1h" }); +} + +describe("adapter registry authority", () => { + test("enumerates every production adapter exactly once", () => { + expect(adapterDefinitions().map(([id]) => id)).toEqual(Object.keys(EXPECTED_ADAPTER_NAMES)); + }); + + test("records semantic inheritance without forcing constructor wrapping", () => { + expect(getAdapterDefinition("azure")?.contractParent).toBe("openai-responses"); + expect(getAdapterDefinition("azure-openai")?.contractParent).toBe("openai-responses"); + expect(getAdapterDefinition("mimo-free")?.contractParent).toBe("openai-chat"); + + expect(effectiveAdapterContract("azure").wire).toBe("openai-responses"); + expect(effectiveAdapterContract("azure-openai").wire).toBe("openai-responses"); + expect(effectiveAdapterContract("mimo-free").wire).toBe("openai-chat"); + expect(effectiveAdapterContract("cursor").mutation).toBe("codex-owned-with-gated-native-fallback"); + }); + + test("constructs every current adapter with its existing observable identity", () => { + for (const [adapterId, expectedName] of Object.entries(EXPECTED_ADAPTER_NAMES)) { + expect(createRegisteredAdapter(provider(adapterId)).name, adapterId).toBe(expectedName); + expect(resolveAdapter(provider(adapterId)).name, adapterId).toBe(expectedName); + } + }); + + test("forwards Anthropic cache retention through registry and server resolution", async () => { + await expectLongCacheRetention(createRegisteredAdapter(provider("anthropic"), { cacheRetention: "long" })); + await expectLongCacheRetention(resolveAdapter(provider("anthropic"), "long")); + }); + + test("rejects unknown persisted adapter ids at the runtime boundary", () => { + for (const adapterId of ["not-a-real-adapter", "__proto__", "constructor"]) { + expect(() => createRegisteredAdapter(provider(adapterId))) + .toThrow(`Unknown adapter: ${adapterId}`); + expect(() => effectiveAdapterContract(adapterId)) + .toThrow(`Unknown adapter: ${adapterId}`); + } + }); + + test("rejects non-string persisted adapter ids before registry lookup", () => { + for (const adapterId of [null, 42, ["azure"]]) { + expect(getAdapterDefinition(adapterId)).toBeUndefined(); + const malformed = { ...provider("anthropic"), adapter: adapterId } as unknown as OcxProviderConfig; + expect(() => createRegisteredAdapter(malformed)).toThrow("Unknown adapter:"); + } + }); +}); diff --git a/tests/adapter-tool-conformance.test.ts b/tests/adapter-tool-conformance.test.ts new file mode 100644 index 0000000000..3e23c13f26 --- /dev/null +++ b/tests/adapter-tool-conformance.test.ts @@ -0,0 +1,441 @@ +import { describe, expect, test } from "bun:test"; +import { + adapterDefinitions, + createRegisteredAdapter, + effectiveAdapterContract, + getAdapterDefinition, + type AdapterWire, +} from "../src/adapters/registry"; +import { resetMimoJwtCache } from "../src/adapters/mimo-free"; +import { bridgeToResponsesSSE } from "../src/bridge"; +import { PROVIDER_REGISTRY } from "../src/providers/registry"; +import { parseRequest } from "../src/responses/parser"; +import { buildToolBridgeMaps } from "../src/server/responses"; +import { MODEL_ADAPTER_OVERRIDE_ALLOWED, type OcxParsedRequest, type OcxProviderConfig } from "../src/types"; +import { TOOL_WIRE_DRIVERS } from "./helpers/adapter-conformance/wire-drivers"; +import { createTestTranslatorBudget } from "./helpers/translator-budget"; + +const PATCH = `*** Begin Patch +*** Add File: conformance-안녕.txt ++quote: "double" ++slash: \\ path ++unicode: 世界 +*** End Patch`; + +const EXEC_DESCRIPTION = + "Run JavaScript. declare const tools: { apply_patch(input: string): Promise<unknown>; };"; + +const WIRE_MODELS: Record<AdapterWire, string> = { + "openai-chat": "grok-4.6", + anthropic: "claude-haiku-4-5", + google: "gemini-3.5-flash", + "command-code": "deepseek/deepseek-v4-flash", + kiro: "claude-sonnet-4.5", + "openai-responses": "deepseek-v4-flash", + cursor: "cursor/auto", +}; + +function providerFixture(adapterId: string, wire: AdapterWire): OcxProviderConfig { + const baseUrls: Record<AdapterWire, string> = { + "openai-chat": "https://api.x.ai/v1", + anthropic: "https://api.anthropic.com", + google: "https://generativelanguage.googleapis.com", + "command-code": "https://api.commandcode.ai", + kiro: "https://runtime.us-east-1.kiro.dev", + "openai-responses": "https://api.deepseek.com", + cursor: "https://api2.cursor.sh", + }; + // Semantic wrappers with provider-specific URL shapes must override the wire-family default here. + const baseUrl = adapterId === "mimo-free" + ? "https://api.xiaomimimo.com/api/free-ai/openai/chat" + : adapterId === "azure" || adapterId === "azure-openai" + ? "https://example.openai.azure.com/openai/v1" + : baseUrls[wire]; + return { + adapter: adapterId, + baseUrl, + authMode: wire === "anthropic" || wire === "command-code" ? "oauth" : "key", + apiKey: wire === "kiro" ? "ksk_test" : "test-key", + defaultMaxOutputTokens: 64_000, + googleMode: "ai-studio", + ...(wire === "openai-responses" ? { responsesPath: "/responses" } : {}), + } satisfies OcxProviderConfig; +} + +function prepareForWire(parsed: OcxParsedRequest, wire: AdapterWire): OcxParsedRequest { + if (wire !== "kiro") return parsed; + return { ...parsed, _kiroAuthContext: { apiRegion: "us-east-1" } }; +} + +function codeModeParsed(wire: AdapterWire): OcxParsedRequest { + const model = WIRE_MODELS[wire]; + return prepareForWire(parseRequest({ + model, + instructions: "Use apply_patch for local file edits.", + input: "Patch the requested file.", + stream: true, + tools: [ + { + type: "custom", + name: "exec", + description: EXEC_DESCRIPTION, + format: { type: "grammar", syntax: "lark" }, + }, + { + type: "function", + name: "wait", + description: "Wait for work.", + parameters: { type: "object", properties: {}, additionalProperties: false }, + }, + ], + }), wire); +} + +function freeformParsed(wire: AdapterWire): OcxParsedRequest { + return prepareForWire(parseRequest({ + model: WIRE_MODELS[wire], + input: "Apply the exact patch.", + stream: true, + tools: [{ type: "custom", name: "apply_patch", description: "Apply a patch" }], + }), wire); +} + +function toolChoiceParsed(wire: AdapterWire, toolChoice?: "none"): OcxParsedRequest { + return prepareForWire(parseRequest({ + model: WIRE_MODELS[wire], + input: "Do not call a tool.", + stream: true, + ...(toolChoice ? { tool_choice: toolChoice } : {}), + tools: [ + { type: "custom", name: "apply_patch", description: "Apply a patch" }, + { + type: "function", + name: "noop", + description: "No operation", + parameters: { type: "object", properties: {}, additionalProperties: false }, + }, + ], + }), wire); +} + +function continuationParsed(wire: AdapterWire): OcxParsedRequest { + return prepareForWire(parseRequest({ + model: WIRE_MODELS[wire], + input: [ + { + type: "message", + role: "user", + content: [{ type: "input_text", text: "Apply the patch exactly." }], + }, + { + type: "custom_tool_call", + id: "ctc_patch", + call_id: "call_continue_patch", + name: "apply_patch", + input: PATCH, + }, + { + type: "custom_tool_call_output", + call_id: "call_continue_patch", + output: "Done!", + }, + { + type: "message", + role: "user", + content: [{ type: "input_text", text: "Continue after patch." }], + }, + ], + stream: true, + tools: [{ type: "custom", name: "apply_patch", description: "Apply a patch" }], + }), wire); +} + +async function withMimoBootstrap<T>(adapterId: string, run: () => Promise<T>): Promise<T> { + if (adapterId !== "mimo-free") return await run(); + const originalFetch = globalThis.fetch; + resetMimoJwtCache(); + let bootstrapCalls = 0; + globalThis.fetch = (async (input: RequestInfo | URL) => { + const url = typeof input === "string" + ? input + : input instanceof URL + ? input.href + : input.url; + if (url !== "https://api.xiaomimimo.com/api/free-ai/bootstrap") { + throw new Error(`mimo-free conformance made an unexpected request: ${url}`); + } + bootstrapCalls++; + return new Response(JSON.stringify({ + jwt: "e30.eyJleHAiOjQxMDI0NDQ4MDB9.x", + }), { + status: 200, + headers: { "content-type": "application/json" }, + }); + }) as typeof fetch; + try { + const result = await run(); + if (bootstrapCalls !== 1) { + throw new Error(`expected one MiMo bootstrap request, got ${bootstrapCalls}`); + } + return result; + } finally { + globalThis.fetch = originalFetch; + resetMimoJwtCache(); + } +} + +async function outbound(adapterId: string, parsed: OcxParsedRequest): Promise<string> { + const contract = effectiveAdapterContract(adapterId); + const adapter = createRegisteredAdapter(providerFixture(adapterId, contract.wire)); + return await withMimoBootstrap(adapterId, () => TOOL_WIRE_DRIVERS[contract.wire].observeOutbound(adapter, parsed)); +} + +function advertisedToolNames(wire: AdapterWire, body: string): string[] { + const parsed = JSON.parse(body) as Record<string, unknown>; + if (wire === "openai-chat") { + const tools = parsed.tools as Array<{ function?: { name?: string } }> | undefined; + return (tools ?? []).flatMap(tool => typeof tool.function?.name === "string" ? [tool.function.name] : []); + } + if (wire === "anthropic" || wire === "openai-responses" || wire === "cursor") { + const tools = parsed.tools as Array<{ name?: string }> | undefined; + return (tools ?? []).flatMap(tool => typeof tool.name === "string" ? [tool.name] : []); + } + if (wire === "google") { + const tools = parsed.tools as Array<{ functionDeclarations?: Array<{ name?: string }> }> | undefined; + return (tools ?? []).flatMap(group => + (group.functionDeclarations ?? []).flatMap(tool => typeof tool.name === "string" ? [tool.name] : [])); + } + if (wire === "command-code") { + const params = parsed.params as { tools?: Array<{ name?: string }> } | undefined; + return (params?.tools ?? []).flatMap(tool => typeof tool.name === "string" ? [tool.name] : []); + } + const state = parsed.conversationState as { + currentMessage?: { + userInputMessage?: { + userInputMessageContext?: { + tools?: Array<{ toolSpecification?: { name?: string } }>; + }; + }; + }; + } | undefined; + const tools = state?.currentMessage?.userInputMessage?.userInputMessageContext?.tools ?? []; + return tools.flatMap(tool => typeof tool.toolSpecification?.name === "string" ? [tool.toolSpecification.name] : []); +} + +function toolCallsDisabled(wire: AdapterWire, body: string): boolean { + if (advertisedToolNames(wire, body).length === 0) return true; + const parsed = JSON.parse(body) as Record<string, unknown>; + if (wire === "openai-chat" || wire === "openai-responses") return parsed.tool_choice === "none"; + if (wire === "anthropic") { + const choice = parsed.tool_choice as { type?: unknown } | undefined; + return choice?.type === "none"; + } + if (wire === "google") { + const config = parsed.toolConfig as { functionCallingConfig?: { mode?: unknown } } | undefined; + return config?.functionCallingConfig?.mode === "NONE"; + } + return false; +} + +function inputFromValue(value: unknown): string | undefined { + if (typeof value === "string") { + try { + const row = JSON.parse(value) as { input?: unknown }; + return typeof row.input === "string" ? row.input : value; + } catch { + return value; + } + } + if (value && typeof value === "object" && !Array.isArray(value)) { + const input = (value as Record<string, unknown>).input; + if (typeof input === "string") return input; + } + return undefined; +} + +function continuationInput(wire: AdapterWire, body: string): string | undefined { + const parsed = JSON.parse(body) as Record<string, unknown>; + if (wire === "openai-chat") { + const messages = parsed.messages as Array<{ tool_calls?: Array<{ function?: { name?: string; arguments?: unknown } }> }> | undefined; + for (const message of messages ?? []) { + for (const call of message.tool_calls ?? []) { + if (call.function?.name?.includes("apply_patch")) return inputFromValue(call.function.arguments); + } + } + return undefined; + } + if (wire === "anthropic") { + const messages = parsed.messages as Array<{ content?: unknown }> | undefined; + for (const message of messages ?? []) { + if (!Array.isArray(message.content)) continue; + for (const block of message.content) { + if (!block || typeof block !== "object" || Array.isArray(block)) continue; + const row = block as Record<string, unknown>; + if (row.type === "tool_use" && typeof row.name === "string" && row.name.includes("apply_patch")) { + return inputFromValue(row.input); + } + } + } + return undefined; + } + if (wire === "google") { + const contents = parsed.contents as Array<{ parts?: Array<{ functionCall?: { name?: string; args?: unknown } }> }> | undefined; + for (const content of contents ?? []) { + for (const part of content.parts ?? []) { + if (part.functionCall?.name?.includes("apply_patch")) return inputFromValue(part.functionCall.args); + } + } + return undefined; + } + if (wire === "command-code") { + const params = parsed.params as { messages?: Array<{ content?: Array<Record<string, unknown>> }> } | undefined; + for (const message of params?.messages ?? []) { + for (const part of message.content ?? []) { + if (part.type === "tool-call" && typeof part.toolName === "string" && part.toolName.includes("apply_patch")) { + return inputFromValue(part.input); + } + } + } + return undefined; + } + if (wire === "kiro") { + const state = parsed.conversationState as { + history?: Array<{ assistantResponseMessage?: { toolUses?: Array<{ name?: string; input?: unknown }> } }>; + currentMessage?: { assistantResponseMessage?: { toolUses?: Array<{ name?: string; input?: unknown }> } }; + } | undefined; + const entries = [...(state?.history ?? []), ...(state?.currentMessage ? [state.currentMessage] : [])]; + for (const entry of entries) { + for (const use of entry.assistantResponseMessage?.toolUses ?? []) { + if (use.name?.includes("apply_patch")) return inputFromValue(use.input); + } + } + return undefined; + } + if (wire === "openai-responses") { + const input = parsed.input as Array<Record<string, unknown>> | undefined; + for (const item of input ?? []) { + if (typeof item.name !== "string" || !item.name.includes("apply_patch")) continue; + if (item.type === "custom_tool_call") return inputFromValue(item.input); + if (item.type === "function_call") return inputFromValue(item.arguments); + } + return undefined; + } + const visit = (value: unknown): string | undefined => { + if (!value || typeof value !== "object") return undefined; + if (Array.isArray(value)) { + for (const item of value) { + const found = visit(item); + if (found !== undefined) return found; + } + return undefined; + } + const row = value as Record<string, unknown>; + if (typeof row.name === "string" && row.name.includes("apply_patch")) { + const found = inputFromValue(row.input ?? row.arguments); + if (found !== undefined) return found; + } + for (const nested of Object.values(row)) { + const found = visit(nested); + if (found !== undefined) return found; + } + return undefined; + }; + return visit(parsed); +} + +function parseResponsesFrames(text: string): Array<{ event?: string; data: Record<string, unknown> }> { + return text.split("\n\n") + .map(frame => frame.trim()) + .filter(frame => frame.length > 0 && frame !== "data: [DONE]") + .map(frame => { + const lines = frame.split("\n"); + const event = lines.find(line => line.startsWith("event: "))?.slice(7); + const data = lines.find(line => line.startsWith("data: "))?.slice(6) ?? "{}"; + return { event, data: JSON.parse(data) as Record<string, unknown> }; + }); +} + +async function restoredStreamInput(adapterId: string, wire: AdapterWire): Promise<string | undefined> { + const driver = TOOL_WIRE_DRIVERS[wire]; + if (!driver.streamingToolCall) return undefined; + const parsed = freeformParsed(wire); + const adapter = createRegisteredAdapter(providerFixture(adapterId, wire)); + const body = await withMimoBootstrap(adapterId, () => driver.observeOutbound(adapter, parsed)); + const wireName = driver.extractWireToolName?.(body, "apply_patch") ?? "apply_patch"; + const maps = buildToolBridgeMaps(parsed); + const bridged = bridgeToResponsesSSE( + adapter.parseStream( + driver.streamingToolCall(wireName, JSON.stringify({ input: PATCH })), + createTestTranslatorBudget(), + ), + parsed.modelId, + maps.toolNsMap, + maps.freeformToolNames, + maps.toolSearchToolNames, + undefined, + 2_000, + { declaredToolNames: maps.declaredToolNames }, + ); + const frames = parseResponsesFrames(await new Response(bridged).text()); + return frames.find(frame => frame.event === "response.custom_tool_call_input.done")?.data.input as string | undefined; +} + +describe("registry-derived routed tool conformance", () => { + test("provider and model-wire configuration ids are registry members", () => { + for (const provider of PROVIDER_REGISTRY) { + expect(getAdapterDefinition(provider.adapter), provider.id).toBeDefined(); + for (const value of Object.values(provider.modelWireDefaults ?? {})) { + const adapterId = typeof value === "string" ? value : value.wire; + expect(getAdapterDefinition(adapterId), `${provider.id}:${adapterId}`).toBeDefined(); + } + } + for (const adapterId of MODEL_ADAPTER_OVERRIDE_ALLOWED) { + expect(getAdapterDefinition(adapterId), adapterId).toBeDefined(); + } + }); + + test("every registered adapter keeps the nested apply_patch helper in its final request", async () => { + for (const [adapterId] of adapterDefinitions()) { + const contract = effectiveAdapterContract(adapterId); + const body = await outbound(adapterId, codeModeParsed(contract.wire)); + const advertised = advertisedToolNames(contract.wire, body); + expect(advertised.some(name => name === "exec" || name.endsWith("_exec")), adapterId).toBe(true); + const normalized = body.replace(/\\n/g, " ").replace(/\s+/g, " "); + expect(normalized, adapterId).toContain("apply_patch(input: string)"); + expect(normalized, adapterId).not.toMatch(/(?:do not|don't|never|must not|cannot|can't)[^.]{0,260}\bapply_patch\b/i); + expect(normalized, adapterId).not.toMatch(/\bapply_patch\b[^.]{0,180}\b(?:forbidden|unavailable|off-limits)\b/i); + } + }); + + test("tool_choice none disables every registered adapter's callable tool surface", async () => { + for (const [adapterId] of adapterDefinitions()) { + const contract = effectiveAdapterContract(adapterId); + const enabledBody = await outbound(adapterId, toolChoiceParsed(contract.wire)); + expect(advertisedToolNames(contract.wire, enabledBody).length, `${adapterId}:enabled`).toBeGreaterThan(0); + const disabledBody = await outbound(adapterId, toolChoiceParsed(contract.wire, "none")); + expect(toolCallsDisabled(contract.wire, disabledBody), adapterId).toBe(true); + } + }); + + test("every parsed streaming wire restores hostile freeform input exactly", async () => { + for (const [adapterId] of adapterDefinitions()) { + const contract = effectiveAdapterContract(adapterId); + const driver = TOOL_WIRE_DRIVERS[contract.wire]; + if (!driver.streamingToolCall) { + // OpenAI Responses is a normal passthrough here and only parses routed compaction; + // Cursor's proprietary runTurn stream has focused parser coverage elsewhere. + expect(["openai-responses", "cursor"]).toContain(contract.wire); + continue; + } + expect(await restoredStreamInput(adapterId, contract.wire), adapterId).toBe(PATCH); + } + }); + + test("every registered adapter replays the exact apply_patch input on continuation", async () => { + for (const [adapterId] of adapterDefinitions()) { + const contract = effectiveAdapterContract(adapterId); + const body = await outbound(adapterId, continuationParsed(contract.wire)); + expect(continuationInput(contract.wire, body), adapterId).toBe(PATCH); + } + }); +}); diff --git a/tests/catalog-input-modality-enum.test.ts b/tests/catalog-input-modality-enum.test.ts index e13e8762d4..cc8bf39b35 100644 --- a/tests/catalog-input-modality-enum.test.ts +++ b/tests/catalog-input-modality-enum.test.ts @@ -1,6 +1,7 @@ -import { describe, expect, test } from "bun:test"; +import { beforeEach, describe, expect, test } from "bun:test"; import { ensureStrictCatalogFields } from "../src/codex/catalog/parsing"; import { catalogHintsFromModelsApiItem } from "../src/codex/catalog/provider-fetch"; +import type { OcxConfig } from "../src/types"; /** * Codex parses `input_modalities` as a closed enum of text | image | audio. A single out-of-enum @@ -57,6 +58,21 @@ describe("catalog input_modalities stay inside the enum Codex accepts", () => { */ describe("custom-model API rejects out-of-enum input modalities", () => { let persistCalls = 0; + // Shared fixture: the PUT/POST handlers mutate and persist the config object they + // receive, so seed requests and their follow-ups must see the SAME object (a fresh + // object per call would discard the seeded default before the follow-up asserts on it). + const fixtureConfig = { + providers: { deepseek: { adapter: "openai-chat", baseUrl: "https://example.invalid/v1" } }, + customModels: [] as Array<{ id: string; provider: string; modelId: string; inputModalities?: string[] }>, + } as unknown as OcxConfig; + + beforeEach(() => { + // Seeded WITH modalities on purpose: a fixture without them would let the + // clear-path test pass against a PUT that ignored the field entirely. + fixtureConfig.customModels = [ + { id: "existing-uuid", provider: "deepseek", modelId: "deepseek-v4", inputModalities: ["text", "image"] }, + ]; + }); async function callCustomModels( method: "POST" | "PUT", @@ -73,14 +89,7 @@ describe("custom-model API rejects out-of-enum input modalities", () => { return handleModelRoutes({ req, url, - config: { - providers: { deepseek: { adapter: "openai-chat", baseUrl: "https://example.invalid/v1" } }, - customModels: [ - // Seeded WITH modalities on purpose: a fixture without them would let the - // clear-path test pass against a PUT that ignored the field entirely. - { id: "existing-uuid", provider: "deepseek", modelId: "deepseek-v4", inputModalities: ["text", "image"] }, - ], - } as unknown as Parameters<typeof handleModelRoutes>[0]["config"], + config: fixtureConfig, // This handler mutates and persists the config object it receives. The // fixture must NEVER reach the process-global OPENCODEX_HOME; that exact bug // replaced a real 41KB provider config with this `existing-uuid` fixture. @@ -163,3 +172,245 @@ describe("custom-model API rejects out-of-enum input modalities", () => { expect(persistCalls).toBe(1); }); }); + +/* + * The same closed-enum argument applies to the reasoning ladder: a label outside the Codex + * ladder (low..ultra) stored through /api/custom-models would surface in a catalog the + * upstream never accepts, and the GUI's effort checkboxes are only as honest as the API + * that validates them. Unlike modalities, an EMPTY ladder is meaningful here — it is the + * explicit "no reasoning" override that hides the effort control (#883) — so `[]` is + * stored, not cleared, and `null` is the only way a PUT restores inheritance. + */ +describe("custom-model API validates reasoning-effort ladders", () => { + let persistCalls = 0; + + async function callCustomModels( + method: "POST" | "PUT", + body: unknown, + pathname = "/api/custom-models", + ): Promise<Response | null> { + const { handleModelRoutes } = await import("../src/server/management/model-routes"); + const url = new URL(`http://127.0.0.1:10199${pathname}`); + const req = new Request(url, { + method, + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(body), + }); + return handleModelRoutes({ + req, + url, + config: { + providers: { deepseek: { adapter: "openai-chat", baseUrl: "https://example.invalid/v1" } }, + customModels: [ + // Seeded WITH a ladder on purpose: the null-clear test needs a stored value to + // remove, and the explicit-empty test needs to prove `[]` is NOT a clear. + { id: "existing-uuid", provider: "deepseek", modelId: "deepseek-v4", reasoningEfforts: ["low", "high"] }, + ], + } as unknown as Parameters<typeof handleModelRoutes>[0]["config"], + deps: { + saveConfigPreservingClaudeCode: () => { persistCalls++; }, + } as Parameters<typeof handleModelRoutes>[0]["deps"], + convergeCodexCatalog: async () => ({ + status: "committed", + changed: false, + degraded: false, + notices: [], + }), + syncClaudeAgentDefsBestEffort: async () => {}, + }); + } + + test("POST refuses an effort outside the Codex ladder, naming the offending value", async () => { + persistCalls = 0; + const res = await callCustomModels("POST", { + provider: "deepseek", + modelId: "deepseek-v5", + reasoningEfforts: ["low", "deep"], + }); + expect(res?.status).toBe(400); + expect((await res!.json() as { error?: string }).error).toContain("deep"); + expect(persistCalls).toBe(0); + }); + + test("POST refuses a non-string member instead of filtering it away", async () => { + persistCalls = 0; + const res = await callCustomModels("POST", { + provider: "deepseek", + modelId: "deepseek-v5", + reasoningEfforts: ["low", 42], + }); + expect(res?.status).toBe(400); + expect((await res!.json() as { error?: string }).error).toContain("strings"); + expect(persistCalls).toBe(0); + }); + + test("POST accepts a valid ladder with a member default and dedupes", async () => { + persistCalls = 0; + const res = await callCustomModels("POST", { + provider: "deepseek", + modelId: "deepseek-v6", + reasoningEfforts: ["low", "high", "high"], + defaultReasoningEffort: "high", + }); + expect(res?.status).toBe(201); + const payload = await res!.json() as { reasoningEfforts?: string[]; defaultReasoningEffort?: string }; + expect(payload.reasoningEfforts).toEqual(["low", "high"]); + expect(payload.defaultReasoningEffort).toBe("high"); + expect(persistCalls).toBe(1); + }); + + test("POST stores an explicit empty ladder as the no-reasoning override", async () => { + persistCalls = 0; + const res = await callCustomModels("POST", { + provider: "deepseek", + modelId: "deepseek-v6", + reasoningEfforts: [], + }); + expect(res?.status).toBe(201); + const payload = await res!.json() as { reasoningEfforts?: string[] }; + expect(payload.reasoningEfforts).toEqual([]); + expect(persistCalls).toBe(1); + }); + + test("POST accepts the none/minimal sentinels, canonicalized first", async () => { + persistCalls = 0; + const res = await callCustomModels("POST", { + provider: "deepseek", + modelId: "deepseek-v6", + reasoningEfforts: ["max", "none", "low", "minimal"], + defaultReasoningEffort: "none", + }); + expect(res?.status).toBe(201); + const payload = await res!.json() as { reasoningEfforts?: string[]; defaultReasoningEffort?: string }; + expect(payload.reasoningEfforts).toEqual(["none", "minimal", "low", "max"]); + expect(payload.defaultReasoningEffort).toBe("none"); + expect(persistCalls).toBe(1); + }); + + test("POST refuses a default effort outside the declared ladder", async () => { + persistCalls = 0; + const res = await callCustomModels("POST", { + provider: "deepseek", + modelId: "deepseek-v6", + reasoningEfforts: ["low", "high"], + defaultReasoningEffort: "max", + }); + expect(res?.status).toBe(400); + expect((await res!.json() as { error?: string }).error).toContain("max"); + expect(persistCalls).toBe(0); + }); + + test("POST refuses a default effort without any ladder", async () => { + persistCalls = 0; + const res = await callCustomModels("POST", { + provider: "deepseek", + modelId: "deepseek-v6", + defaultReasoningEffort: "high", + }); + expect(res?.status).toBe(400); + expect((await res!.json() as { error?: string }).error).toContain("reasoningEfforts"); + expect(persistCalls).toBe(0); + }); + + test("PUT stores an explicit empty ladder instead of clearing it", async () => { + persistCalls = 0; + const res = await callCustomModels("PUT", { reasoningEfforts: [] }, "/api/custom-models/existing-uuid"); + expect(res?.status).toBe(200); + const payload = await res!.json() as { reasoningEfforts?: string[] }; + expect(payload.reasoningEfforts).toEqual([]); + expect(persistCalls).toBe(1); + }); + + test("PUT null restores inheritance by clearing the stored ladder", async () => { + persistCalls = 0; + const res = await callCustomModels("PUT", { reasoningEfforts: null }, "/api/custom-models/existing-uuid"); + expect(res?.status).toBe(200); + const payload = await res!.json() as { reasoningEfforts?: string[] }; + expect(payload.reasoningEfforts).toBeUndefined(); + expect(persistCalls).toBe(1); + }); + + test("PUT clears the default when the ladder is removed", async () => { + persistCalls = 0; + const res = await callCustomModels( + "PUT", + { reasoningEfforts: null, defaultReasoningEffort: null }, + "/api/custom-models/existing-uuid", + ); + expect(res?.status).toBe(200); + const payload = await res!.json() as { reasoningEfforts?: string[]; defaultReasoningEffort?: string }; + expect(payload.reasoningEfforts).toBeUndefined(); + expect(payload.defaultReasoningEffort).toBeUndefined(); + expect(persistCalls).toBe(1); + }); + + // POST rejects a default outside the ladder; PUT must not be able to produce that state + // on its own. A ladder shrink/clear on a row that was created with a default (CLI) must + // drop the stale default — otherwise it re-applies itself onto the inherited ladder in + // the generated catalog (GUI toggle-off path sends only reasoningEfforts). + test("PUT ladder shrink drops a stored default that is no longer a member", async () => { + persistCalls = 0; + const seededRes = await callCustomModels("PUT", { + reasoningEfforts: ["low", "high", "max"], + defaultReasoningEffort: "max", + }, "/api/custom-models/existing-uuid"); + expect(seededRes?.status).toBe(200); + const seeded = await seededRes!.json() as { defaultReasoningEffort?: string }; + expect(seeded.defaultReasoningEffort).toBe("max"); + + persistCalls = 0; + const res = await callCustomModels("PUT", { reasoningEfforts: ["low"] }, "/api/custom-models/existing-uuid"); + expect(res?.status).toBe(200); + const payload = await res!.json() as { reasoningEfforts?: string[]; defaultReasoningEffort?: string }; + expect(payload.reasoningEfforts).toEqual(["low"]); + expect(payload.defaultReasoningEffort).toBeUndefined(); + expect(persistCalls).toBe(1); + }); + + test("PUT null-clear drops a stored default even when the body does not mention it", async () => { + persistCalls = 0; + const seededRes = await callCustomModels("PUT", { + reasoningEfforts: ["low", "high"], + defaultReasoningEffort: "high", + }, "/api/custom-models/existing-uuid"); + expect(seededRes?.status).toBe(200); + + persistCalls = 0; + const res = await callCustomModels("PUT", { reasoningEfforts: null }, "/api/custom-models/existing-uuid"); + expect(res?.status).toBe(200); + const payload = await res!.json() as { reasoningEfforts?: string[]; defaultReasoningEffort?: string }; + expect(payload.reasoningEfforts).toBeUndefined(); + expect(payload.defaultReasoningEffort).toBeUndefined(); + expect(persistCalls).toBe(1); + }); + + test("PUT explicit empty ladder also drops a stored default", async () => { + persistCalls = 0; + const seededRes = await callCustomModels("PUT", { + reasoningEfforts: ["low", "high"], + defaultReasoningEffort: "high", + }, "/api/custom-models/existing-uuid"); + expect(seededRes?.status).toBe(200); + + persistCalls = 0; + const res = await callCustomModels("PUT", { reasoningEfforts: [] }, "/api/custom-models/existing-uuid"); + expect(res?.status).toBe(200); + const payload = await res!.json() as { reasoningEfforts?: string[]; defaultReasoningEffort?: string }; + expect(payload.reasoningEfforts).toEqual([]); + expect(payload.defaultReasoningEffort).toBeUndefined(); + expect(persistCalls).toBe(1); + }); + + test("POST and PUT canonicalize the ladder into Codex order", async () => { + persistCalls = 0; + const res = await callCustomModels("POST", { + provider: "deepseek", + modelId: "deepseek-v6", + reasoningEfforts: ["max", "low", "high", "low"], + }); + expect(res?.status).toBe(201); + const payload = await res!.json() as { reasoningEfforts?: string[] }; + expect(payload.reasoningEfforts).toEqual(["low", "high", "max"]); + expect(persistCalls).toBe(1); + }); +}); diff --git a/tests/cli-models-reasoning.test.ts b/tests/cli-models-reasoning.test.ts new file mode 100644 index 0000000000..4c7e131520 --- /dev/null +++ b/tests/cli-models-reasoning.test.ts @@ -0,0 +1,174 @@ +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { parseReasoningArgs, handleModels } from "../src/cli/models"; +import { handleModelsRuntimeCommand } from "../src/cli/models-runtime"; + +/** + * The API validates reasoning ladders (9 tests in catalog-input-modality-enum.test.ts), + * but the CLI paths carry their own parsing and validation copies: `ocx models add` + * validates offline before writing config.json, and `ocx models edit` maps flags onto + * the PUT body ("-" -> null). These tests pin that mapping so CLI and API cannot drift. + */ +describe("ocx models add --reasoning-efforts parsing", () => { + test("a valid ladder is canonicalized into Codex order and deduped", () => { + expect(parseReasoningArgs("max,low,high,low", undefined)).toEqual({ + reasoningEfforts: ["low", "high", "max"], + }); + }); + + test("the none sentinel is accepted and canonicalized first", () => { + expect(parseReasoningArgs("low,none,max", undefined)).toEqual({ + reasoningEfforts: ["none", "low", "max"], + }); + }); + + test("an unknown effort is rejected and names the offending value", () => { + const parsed = parseReasoningArgs("low,deep", undefined); + expect(parsed.error).toContain("deep"); + expect(parsed.reasoningEfforts).toBeUndefined(); + }); + + test('an empty string is the explicit no-reasoning ladder; malformed CSV is rejected', () => { + expect(parseReasoningArgs("", undefined)).toEqual({ reasoningEfforts: [] }); + expect(parseReasoningArgs("low,,high", undefined)?.error).toContain("comma-separated"); + expect(parseReasoningArgs(",,", undefined)?.error).toContain("comma-separated"); + }); + + test("a default still cannot ride on an explicit empty ladder", () => { + expect(parseReasoningArgs("", "low")?.error).toContain("requires --reasoning-efforts"); + }); + + test('"-" omits the field (inherit) exactly like the API null-clear', () => { + expect(parseReasoningArgs("-", undefined)).toEqual({}); + expect(parseReasoningArgs(undefined, "-")).toEqual({}); + }); + + test("a default must be a ladder member", () => { + const parsed = parseReasoningArgs("low,high", "max"); + expect(parsed.error).toContain("max"); + expect(parsed.error).toContain("not in the declared reasoning efforts"); + }); + + test("a default requires a ladder", () => { + expect(parseReasoningArgs(undefined, "high")?.error).toContain("requires --reasoning-efforts"); + }); + + test("a member default is accepted", () => { + expect(parseReasoningArgs("low,high", "high")).toEqual({ + reasoningEfforts: ["low", "high"], + defaultReasoningEffort: "high", + }); + }); +}); + +describe("ocx models edit reasoning flag mapping onto the PUT body", () => { + async function editWith(patchArgs: string[]): Promise<Record<string, unknown>> { + let capturedBody: Record<string, unknown> | null = null; + const fetchImpl = async (url: string, init?: RequestInit) => { + capturedBody = JSON.parse(String(init?.body)); + return new Response(JSON.stringify({ id: "cm-1", ...capturedBody }), { + status: 200, + headers: { "Content-Type": "application/json" }, + }); + }; + const code = await handleModelsRuntimeCommand("edit", ["cm-1", ...patchArgs], { + baseUrl: "http://127.0.0.1:1", + fetchImpl, + }); + expect(code).toBe(0); + return capturedBody ?? {}; + } + + test('"--reasoning-efforts -" maps to null (restore inheritance)', async () => { + const body = await editWith(["--reasoning-efforts", "-"]); + expect(body.reasoningEfforts).toBeNull(); + }); + + test('"--reasoning-efforts \"\"" stores an explicit empty ladder (no-reasoning override)', async () => { + const body = await editWith(["--reasoning-efforts", ""]); + expect(body.reasoningEfforts).toEqual([]); + }); + + test("embedded blank CSV members are rejected without touching the API", async () => { + let fetchCalled = false; + const fetchImpl = async () => { fetchCalled = true; return new Response("{}", { status: 200 }); }; + const code = await handleModelsRuntimeCommand("edit", ["cm-1", "--reasoning-efforts", "low,,high"], { + baseUrl: "http://127.0.0.1:1", + fetchImpl, + }); + // runCliAction turns CliUsageError into exit code 2 without touching the API. + expect(code).toBe(2); + expect(fetchCalled).toBe(false); + }); + + test("a csv ladder maps to an array", async () => { + const body = await editWith(["--reasoning-efforts", "low,high"]); + expect(body.reasoningEfforts).toEqual(["low", "high"]); + }); + + test('"--default-reasoning-effort -" maps to null', async () => { + const body = await editWith(["--default-reasoning-effort", "-"]); + expect(body.defaultReasoningEffort).toBeNull(); + }); + + test("a member default maps to its string", async () => { + const body = await editWith(["--reasoning-efforts", "low,high", "--default-reasoning-effort", "high"]); + expect(body.reasoningEfforts).toEqual(["low", "high"]); + expect(body.defaultReasoningEffort).toBe("high"); + }); +}); + +describe("ocx models add persists reasoning metadata into config.json", () => { + const home = mkdtempSync(join(tmpdir(), "ocx-cli-test-")); + const previousHome = process.env.OPENCODEX_HOME; + + beforeAll(() => { + process.env.OPENCODEX_HOME = home; + writeFileSync(join(home, "config.json"), JSON.stringify({ + providers: { + deepseek: { adapter: "openai-chat", baseUrl: "https://example.invalid/v1", authMode: "key" }, + }, + })); + }); + + afterAll(() => { + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + rmSync(home, { recursive: true, force: true }); + }); + + function readConfig(): { customModels?: Array<Record<string, unknown>> } { + return JSON.parse(readFileSync(join(home, "config.json"), "utf8")); + } + + test("a ladder with a member default is stored canonicalized", async () => { + await handleModels(["add", "deepseek", "m1", "--reasoning-efforts", "max,low,high", "--default-reasoning-effort", "high"]); + const entry = readConfig().customModels!.find(model => model.modelId === "m1")!; + expect(entry.reasoningEfforts).toEqual(["low", "high", "max"]); + expect(entry.defaultReasoningEffort).toBe("high"); + }); + + test('"-" omits the reasoning fields entirely (inherit)', async () => { + await handleModels(["add", "deepseek", "m2", "--reasoning-efforts", "-"]); + const entry = readConfig().customModels!.find(model => model.modelId === "m2")!; + expect(entry.reasoningEfforts).toBeUndefined(); + expect(entry.defaultReasoningEffort).toBeUndefined(); + }); + + test("list-custom renders the stored ladder columns", async () => { + const lines: string[] = []; + const originalLog = console.log; + console.log = (...args: unknown[]) => { lines.push(args.map(String).join(" ")); }; + try { + await handleModels(["list-custom"]); + } finally { + console.log = originalLog; + } + const table = lines.join("\n"); + expect(table).toContain("EFFORTS"); + expect(table).toContain("low,high,max"); + expect(table).toContain("-"); // m2 has no ladder -> dash cell + }); +}); diff --git a/tests/client-config-export.test.ts b/tests/client-config-export.test.ts index be8491924c..a9eaaff321 100644 --- a/tests/client-config-export.test.ts +++ b/tests/client-config-export.test.ts @@ -182,10 +182,56 @@ describe("Pi serializer (accept criterion 2)", () => { expect(JSON.stringify(piConfig())).not.toContain("cost"); }); - test("reasoning is omitted — an effort list is not Pi's boolean", () => { + test("reasoning is emitted only for rows with a non-empty effort ladder", () => { + // The shared fixture carries no ladder anywhere: every entry stays reasoning-free. for (const model of piConfig().providers.opencodex!.models) { expect(model).not.toHaveProperty("reasoning"); } + const config = piConfig(ctx({ + models: [ + { namespaced: "a/reasoning", provider: "a", id: "reasoning", reasoningEfforts: ["low", "high"] }, + { namespaced: "b/none", provider: "b", id: "none", reasoningEfforts: [] }, + { namespaced: "c/plain", provider: "c", id: "plain" }, + { namespaced: "d/off", provider: "d", id: "off", reasoningEfforts: ["none", "minimal", "low"] }, + ], + })); + const models = config.providers.opencodex!.models; + expect(models.find(model => model.id === "a/reasoning")!.reasoning).toBe(true); + // Pi's level scale is constrained to the ladder: members map to themselves, everything + // else (incl. minimal, which the Codex ladder has no equivalent for) is hidden. + expect(models.find(model => model.id === "a/reasoning")!.thinkingLevelMap).toEqual({ + off: null, + minimal: null, + low: "low", + medium: null, + high: "high", + xhigh: null, + max: null, + }); + // The none sentinel maps pi's off level to "none" (the proxy omits the parameter); + // minimal maps to itself. + expect(models.find(model => model.id === "d/off")!.thinkingLevelMap).toEqual({ + off: "none", + minimal: "minimal", + low: "low", + medium: null, + high: null, + xhigh: null, + max: null, + }); + // An ultra-only ladder has no exact pi level: pi's max maps to the only declared tier + // so the model's sole reasoning level is actually selectable. + const ultraOnly = piConfig(ctx({ + models: [{ namespaced: "e/ultra", provider: "e", id: "ultra", reasoningEfforts: ["ultra"] }], + })); + expect(ultraOnly.providers.opencodex!.models[0]!.thinkingLevelMap).toMatchObject({ + max: "ultra", + off: null, + minimal: null, + }); + // An explicit empty ladder is the catalog's "no reasoning" statement; no boolean. + expect(models.find(model => model.id === "b/none")).not.toHaveProperty("reasoning"); + expect(models.find(model => model.id === "c/plain")).not.toHaveProperty("reasoning"); }); test("contextWindow and maxTokens are omitted when the context window is unknown", () => { diff --git a/tests/cline-pass-provider.test.ts b/tests/cline-pass-provider.test.ts index a6b8ab3725..c5e963731e 100644 --- a/tests/cline-pass-provider.test.ts +++ b/tests/cline-pass-provider.test.ts @@ -18,6 +18,7 @@ const OFFICIAL_CLINE_PASS_MODELS = [ "cline-pass/mimo-v2.5", "cline-pass/mimo-v2.5-pro", "cline-pass/minimax-m3", + "cline-pass/qwen3.8-max", "cline-pass/qwen3.7-max", "cline-pass/qwen3.7-plus", ]; @@ -76,6 +77,8 @@ describe("ClinePass provider", () => { "cline-pass/mimo-v2.5-pro", "cline-pass/qwen3.7-max", ]); + expect(entry?.modelContextWindows?.["cline-pass/qwen3.8-max"]).toBeUndefined(); + expect(entry?.modelInputModalities?.["cline-pass/qwen3.8-max"]).toBeUndefined(); expect(entry?.modelInputModalities?.["cline-pass/kimi-k3"]).toEqual(["text", "image"]); expect(entry?.modelInputModalities?.["cline-pass/glm-5.2"]).toEqual(["text"]); expect(KEY_LOGIN_PROVIDERS["cline-pass"]?.models).toEqual(OFFICIAL_CLINE_PASS_MODELS); diff --git a/tests/codex-catalog-sync-hardening.test.ts b/tests/codex-catalog-sync-hardening.test.ts index ed48d5b855..7a359079f5 100644 --- a/tests/codex-catalog-sync-hardening.test.ts +++ b/tests/codex-catalog-sync-hardening.test.ts @@ -907,7 +907,7 @@ describe("Codex catalog sync hardening", () => { expect(slugs).not.toContain("offline/disabled-model"); expect(slugs).not.toContain("removed/ghost"); expect(slugs).toContain("cursor/composer-2.5"); - }); + }, 15_000); test("drops legacy-signature ghost rows in both gather branches", () => { const catalogPath = join(codexHome, "catalog.json"); diff --git a/tests/codex-catalog.test.ts b/tests/codex-catalog.test.ts index 3ebcfe85a5..60021106f4 100644 --- a/tests/codex-catalog.test.ts +++ b/tests/codex-catalog.test.ts @@ -1704,8 +1704,10 @@ describe("Google Gemini catalog metadata", () => { const entry = buildCatalogEntries(nativeTemplate(), [], models) .find(row => row.slug === "google/gemini-3.6-flash"); + // The registry ladder declares minimal for Gemini 3.6 Flash; it now flows through + // (previously sanitize silently dropped it) plus the mock top rungs for subagent spawns. expect((entry?.supported_reasoning_levels as Array<{ effort: string }>).map(level => level.effort)) - .toEqual(["low", "medium", "high", "max", "ultra"]); + .toEqual(["minimal", "low", "medium", "high", "max", "ultra"]); expect(entry?.input_modalities).toEqual(["text", "image"]); expect(entry?.context_window).toBe(1_048_576); }); @@ -1877,6 +1879,243 @@ describe("configured CatalogModel displayName -> catalog display_name", () => { clearModelCache("custom-provider"); } }); + + test("a customModel reasoning ladder overrides the inherited provider ladder end-to-end", async () => { + clearModelCache("custom-provider"); + const originalFetch = globalThis.fetch; + let fetchCalls = 0; + globalThis.fetch = (() => { + fetchCalls += 1; + throw new Error("fetch should not be called"); + }) as typeof fetch; + try { + const models = await gatherRoutedModels({ + port: 10100, + defaultProvider: "custom-provider", + providers: { + "custom-provider": { + baseUrl: "https://example.invalid/v1", + adapter: "openai-chat", + authMode: "key", + liveModels: false, + models: ["baseline-model", "renamed-model"], + // The provider row for the same slug advertises low/high; the custom row must win. + modelReasoningEfforts: { "baseline-model": ["low", "high"], "renamed-model": ["low", "high"] }, + }, + }, + customModels: [ + { + id: "cm-1", + provider: "custom-provider", + modelId: "renamed-model", + displayName: "Renamed Model", + reasoningEfforts: ["medium", "max"], + defaultReasoningEffort: "max", + addedAt: "2026-01-01T00:00:00.000Z", + }, + ], + }); + + expect(fetchCalls).toBe(0); + const custom = models.find(m => m.provider === "custom-provider" && m.id === "renamed-model"); + // The explicit ladder rides on the row itself, not on the replaced provider row. + expect(custom?.reasoningEfforts).toEqual(["medium", "max"]); + expect(custom?.defaultReasoningEffort).toBe("max"); + + const entries = buildCatalogEntries(nativeTemplate(), [], models); + const row = entries.find(e => e.slug === "custom-provider/renamed-model"); + const levels = (row?.supported_reasoning_levels ?? []).map((l: { effort: string }) => l.effort); + // The sync appends the mock top rungs (max/ultra) for subagent spawn compatibility; + // the declared medium/max survive verbatim, the inherited low/high does not. + expect(levels).toEqual(["medium", "max", "ultra"]); + expect(row?.default_reasoning_level).toBe("max"); + } finally { + globalThis.fetch = originalFetch; + clearModelCache("custom-provider"); + } + }); + + test("an explicit empty customModel ladder hides the effort control despite an inherited one", async () => { + clearModelCache("custom-provider"); + const originalFetch = globalThis.fetch; + globalThis.fetch = (() => { + throw new Error("fetch should not be called"); + }) as typeof fetch; + try { + const models = await gatherRoutedModels({ + port: 10100, + defaultProvider: "custom-provider", + providers: { + "custom-provider": { + baseUrl: "https://example.invalid/v1", + adapter: "openai-chat", + authMode: "key", + liveModels: false, + models: ["renamed-model"], + modelReasoningEfforts: { "renamed-model": ["low", "high"] }, + }, + }, + customModels: [ + { + id: "cm-1", + provider: "custom-provider", + modelId: "renamed-model", + reasoningEfforts: [], + addedAt: "2026-01-01T00:00:00.000Z", + }, + ], + }); + + const custom = models.find(m => m.provider === "custom-provider" && m.id === "renamed-model"); + expect(custom?.reasoningEfforts).toEqual([]); + + const entries = buildCatalogEntries(nativeTemplate(), [], models); + const row = entries.find(e => e.slug === "custom-provider/renamed-model"); + expect(row?.supported_reasoning_levels).toEqual([]); + expect(row).not.toHaveProperty("default_reasoning_level"); + } finally { + globalThis.fetch = originalFetch; + clearModelCache("custom-provider"); + } + }); + + test("a none-only custom ladder advertises no synthetic top rungs", async () => { + clearModelCache("custom-provider"); + const originalFetch = globalThis.fetch; + globalThis.fetch = (() => { + throw new Error("fetch should not be called"); + }) as typeof fetch; + try { + const models = await gatherRoutedModels({ + port: 10100, + defaultProvider: "custom-provider", + providers: { + "custom-provider": { + baseUrl: "https://example.invalid/v1", + adapter: "openai-chat", + authMode: "key", + liveModels: false, + models: ["renamed-model"], + }, + }, + customModels: [ + { + id: "cm-1", + provider: "custom-provider", + modelId: "renamed-model", + reasoningEfforts: ["none"], + addedAt: "2026-01-01T00:00:00.000Z", + }, + ], + }); + + const custom = models.find(m => m.provider === "custom-provider" && m.id === "renamed-model"); + expect(custom?.reasoningEfforts).toEqual(["none"]); + + const entries = buildCatalogEntries(nativeTemplate(), [], models); + const row = entries.find(e => e.slug === "custom-provider/renamed-model"); + const levels = (row?.supported_reasoning_levels ?? []).map((l: { effort: string }) => l.effort); + // No reasoning-capable rung -> the mock max/ultra repair must not fire. + expect(levels).toEqual(["none"]); + expect(row?.default_reasoning_level).toBe("none"); + } finally { + globalThis.fetch = originalFetch; + clearModelCache("custom-provider"); + } + }); + + test("a mixed none+low custom ladder keeps none first and gets the mock top rungs", async () => { + clearModelCache("custom-provider"); + const originalFetch = globalThis.fetch; + globalThis.fetch = (() => { + throw new Error("fetch should not be called"); + }) as typeof fetch; + try { + const models = await gatherRoutedModels({ + port: 10100, + defaultProvider: "custom-provider", + providers: { + "custom-provider": { + baseUrl: "https://example.invalid/v1", + adapter: "openai-chat", + authMode: "key", + liveModels: false, + models: ["renamed-model"], + }, + }, + customModels: [ + { + id: "cm-1", + provider: "custom-provider", + modelId: "renamed-model", + reasoningEfforts: ["none", "low"], + addedAt: "2026-01-01T00:00:00.000Z", + }, + ], + }); + + const entries = buildCatalogEntries(nativeTemplate(), [], models); + const row = entries.find(e => e.slug === "custom-provider/renamed-model"); + const levels = (row?.supported_reasoning_levels ?? []).map((l: { effort: string }) => l.effort); + expect(levels).toEqual(["none", "low", "max", "ultra"]); + // `none` is declared but real rungs exist: the implicit default must be low, not none. + expect(row?.default_reasoning_level).toBe("low"); + } finally { + globalThis.fetch = originalFetch; + clearModelCache("custom-provider"); + } + }); + + test("an inherited provider default does not ride onto a custom ladder that excludes it", async () => { + clearModelCache("custom-provider"); + const originalFetch = globalThis.fetch; + globalThis.fetch = (() => { + throw new Error("fetch should not be called"); + }) as typeof fetch; + try { + const models = await gatherRoutedModels({ + port: 10100, + defaultProvider: "custom-provider", + providers: { + "custom-provider": { + baseUrl: "https://example.invalid/v1", + adapter: "openai-chat", + authMode: "key", + liveModels: false, + models: ["renamed-model"], + // The provider row advertises low/high with a high default; the custom ladder + // drops high, so the merged row must not keep advertising high as default. + modelReasoningEfforts: { "renamed-model": ["low", "high"] }, + modelDefaultReasoningEfforts: { "renamed-model": "high" }, + }, + }, + customModels: [ + { + id: "cm-1", + provider: "custom-provider", + modelId: "renamed-model", + reasoningEfforts: ["low"], + addedAt: "2026-01-01T00:00:00.000Z", + }, + ], + }); + + const custom = models.find(m => m.provider === "custom-provider" && m.id === "renamed-model"); + expect(custom?.reasoningEfforts).toEqual(["low"]); + expect(custom?.defaultReasoningEffort).toBeUndefined(); + + const entries = buildCatalogEntries(nativeTemplate(), [], models); + const row = entries.find(e => e.slug === "custom-provider/renamed-model"); + const levels = (row?.supported_reasoning_levels ?? []).map((l: { effort: string }) => l.effort); + expect(levels).toEqual(["low", "max", "ultra"]); + // No high in the ladder, so the fallback default is medium? low is the first rung — + // applyReasoningLevels picks medium when present, else high, else the first entry. + expect(row?.default_reasoning_level).toBe("low"); + } finally { + globalThis.fetch = originalFetch; + clearModelCache("custom-provider"); + } + }); }); describe("legacy custom-model catalog ownership", () => { @@ -2676,6 +2915,48 @@ describe("Codex catalog routed normalization", () => { expect(row?.multi_agent_version).toBeUndefined(); }); + test("an explicit empty custom ladder beats the native-alias ladder on a forward row", async () => { + const originalFetch = globalThis.fetch; + globalThis.fetch = (() => { throw new Error("forward providers must not fetch /models"); }) as typeof fetch; + try { + const models = await gatherRoutedModels({ + port: 10100, + defaultProvider: "openai", + providers: { + openai: { + adapter: "openai-responses", + baseUrl: "https://chatgpt.com/backend-api/codex", + codexAccountMode: "pool", + }, + }, + codexAccountPickerEnabled: false, + codexAccountNamespaces: { main: "@main" }, + customModels: [{ + id: "daybreak-no-reasoning", + provider: "openai", + modelId: NATIVE_DAYBREAK_BLUE_MODEL, + // Explicit "no reasoning": the alias's native ladder (low..ultra, default low) must + // not overwrite it — otherwise the catalog would advertise reasoning the user + // explicitly disabled for this row. + reasoningEfforts: [], + }], + }); + const model = models.find(row => row.provider === "openai" && row.id === NATIVE_DAYBREAK_BLUE_MODEL); + expect(model).toMatchObject({ + codexForwardNativeCapabilityAlias: true, + reasoningEfforts: [], + }); + expect(model?.defaultReasoningEffort).toBeUndefined(); + + const entries = buildCatalogEntries(nativeTemplate(), [], models); + const daybreak = entries.find(entry => entry.slug === `openai/${NATIVE_DAYBREAK_BLUE_MODEL}`); + expect(daybreak?.supported_reasoning_levels).toEqual([]); + expect(daybreak).not.toHaveProperty("default_reasoning_level"); + } finally { + globalThis.fetch = originalFetch; + } + }); + test("catalog sync upgrades fallback-quality gpt-5.6 entries but preserves genuine ones", () => { // Fallback-quality: display_name stamped with the bare slug (ocx synthesis signature), // wrong ladder (ultra on luna) left by an older ocx version. diff --git a/tests/helpers/adapter-conformance/wire-drivers.ts b/tests/helpers/adapter-conformance/wire-drivers.ts new file mode 100644 index 0000000000..3abc7855eb --- /dev/null +++ b/tests/helpers/adapter-conformance/wire-drivers.ts @@ -0,0 +1,282 @@ +import { create, fromBinary } from "@bufbuild/protobuf"; +import type { ProviderAdapter } from "../../../src/adapters/base"; +import type { AdapterWire } from "../../../src/adapters/registry"; +import { decodeCursorArgsMap } from "../../../src/adapters/cursor/arg-codec"; +import { + AgentClientMessageSchema, + ConversationStepSchema, + ConversationTurnStructureSchema, + GetBlobArgsSchema, + KvServerMessageSchema, +} from "../../../src/adapters/cursor/gen/agent_pb"; +import { + handleCursorNativeKv, + releaseCursorBlobRequestScope, + type CursorBlobRequestScopeToken, +} from "../../../src/adapters/cursor/native-exec"; +import { prepareCursorRunRequest } from "../../../src/adapters/cursor/protobuf-request"; +import { createCursorRequest } from "../../../src/adapters/cursor/request-builder"; +import { encodeMessage } from "../../../src/lib/eventstream-decoder"; +import type { OcxParsedRequest } from "../../../src/types"; +import { withTestTranslatorBudget } from "../translator-budget"; + +export interface ToolWireDriver { + observeOutbound(adapter: ProviderAdapter, parsed: OcxParsedRequest): Promise<string>; + extractWireToolName?(body: string, canonicalName: string): string; + streamingToolCall?(wireName: string, wrappedArguments: string): Response; +} + +async function observeHttpOutbound(adapter: ProviderAdapter, parsed: OcxParsedRequest): Promise<string> { + const testAdapter = withTestTranslatorBudget(adapter); + const request = await testAdapter.buildRequest(parsed); + try { + return request.body; + } finally { + request.releaseBodyObservation?.(); + } +} + +function cursorBlobData(blobId: Uint8Array, scope: CursorBlobRequestScopeToken): Uint8Array { + const reply = fromBinary(AgentClientMessageSchema, handleCursorNativeKv(create(KvServerMessageSchema, { + id: 1, + message: { case: "getBlobArgs", value: create(GetBlobArgsSchema, { blobId }) }, + }), scope)); + if (reply.message.case !== "kvClientMessage") { + throw new Error(`Cursor conformance expected kvClientMessage, got ${reply.message.case || "empty"}`); + } + const kv = reply.message.value; + if (kv.message.case !== "getBlobResult" || !kv.message.value.blobData) { + throw new Error(`Cursor conformance could not hydrate blob ${Buffer.from(blobId).toString("hex")}`); + } + return kv.message.value.blobData; +} + +function splitInTwo(input: string): [string, string] { + const split = Math.max(1, Math.floor(input.length / 2)); + return [input.slice(0, split), input.slice(split)]; +} + +function openAiChatToolCall(wireName: string, wrappedArguments: string): Response { + const fragments = splitInTwo(wrappedArguments); + const frames = fragments.map((argumentsFragment, index) => ({ + choices: [{ + delta: { + tool_calls: [{ + index: 0, + ...(index === 0 ? { id: "call_patch", type: "function" } : {}), + function: { + ...(index === 0 ? { name: wireName } : {}), + arguments: argumentsFragment, + }, + }], + }, + finish_reason: index === fragments.length - 1 ? "tool_calls" : null, + }], + })); + return new Response(`${frames.map(frame => `data: ${JSON.stringify(frame)}`).join("\n\n")}\n\ndata: [DONE]\n\n`, { + headers: { "content-type": "text/event-stream" }, + }); +} + +function anthropicToolCall(wireName: string, wrappedArguments: string): Response { + const fragments = splitInTwo(wrappedArguments); + const frame = (event: string, data: unknown) => `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`; + return new Response([ + frame("content_block_start", { + type: "content_block_start", + content_block: { type: "tool_use", id: "toolu_patch", name: wireName }, + }), + ...fragments.map(partialJson => frame("content_block_delta", { + type: "content_block_delta", + delta: { type: "input_json_delta", partial_json: partialJson }, + })), + frame("content_block_stop", { type: "content_block_stop" }), + frame("message_stop", { type: "message_stop" }), + ].join(""), { headers: { "content-type": "text/event-stream" } }); +} + +function googleToolCall(wireName: string, wrappedArguments: string): Response { + return new Response( + `data: ${JSON.stringify({ + candidates: [{ + content: { parts: [{ functionCall: { name: wireName, args: JSON.parse(wrappedArguments) } }] }, + finishReason: "STOP", + }], + })}\n\n`, + { headers: { "content-type": "text/event-stream" } }, + ); +} + +function commandCodeToolCall(wireName: string, wrappedArguments: string): Response { + return new Response([ + JSON.stringify({ + type: "tool-call", + toolCallId: "call_patch", + toolName: wireName, + input: JSON.parse(wrappedArguments), + }), + JSON.stringify({ type: "finish", rawFinishReason: "tool_use" }), + ].join("\n")); +} + +const kiroEncoder = new TextEncoder(); +function kiroFrame(payload: unknown): Uint8Array { + return encodeMessage( + { ":message-type": "event", ":event-type": "toolUseEvent" }, + kiroEncoder.encode(JSON.stringify(payload)), + ); +} + +function kiroToolCall(wireName: string, wrappedArguments: string): Response { + const fragments = splitInTwo(wrappedArguments); + const frames = [ + kiroFrame({ name: wireName, toolUseId: "call_patch" }), + ...fragments.map(input => kiroFrame({ input, name: wireName, toolUseId: "call_patch" })), + kiroFrame({ name: wireName, stop: true, toolUseId: "call_patch" }), + ]; + let index = 0; + return new Response(new ReadableStream<Uint8Array>({ + pull(controller) { + if (index < frames.length) controller.enqueue(frames[index++]!); + else controller.close(); + }, + })); +} + +function requireWireToolName( + match: string | undefined, + canonicalName: string, + wire: AdapterWire, +): string { + if (!match) throw new Error(`${wire} outbound body advertised no tool matching "${canonicalName}"`); + return match; +} + +const openAiChatDriver: ToolWireDriver = { + observeOutbound: observeHttpOutbound, + extractWireToolName(body, canonicalName) { + const parsed = JSON.parse(body) as { tools?: Array<{ function?: { name?: string } }> }; + const match = parsed.tools?.find(tool => tool.function?.name?.includes(canonicalName))?.function?.name; + return requireWireToolName(match, canonicalName, "openai-chat"); + }, + streamingToolCall: openAiChatToolCall, +}; + +const anthropicDriver: ToolWireDriver = { + observeOutbound: observeHttpOutbound, + extractWireToolName(body, canonicalName) { + const parsed = JSON.parse(body) as { tools?: Array<{ name?: string }> }; + const match = parsed.tools?.find(tool => tool.name?.includes(canonicalName))?.name; + return requireWireToolName(match, canonicalName, "anthropic"); + }, + streamingToolCall: anthropicToolCall, +}; + +const googleDriver: ToolWireDriver = { + observeOutbound: observeHttpOutbound, + extractWireToolName(body, canonicalName) { + const parsed = JSON.parse(body) as { + tools?: Array<{ functionDeclarations?: Array<{ name?: string }> }>; + }; + for (const toolGroup of parsed.tools ?? []) { + const match = toolGroup.functionDeclarations?.find(tool => tool.name?.includes(canonicalName))?.name; + if (match) return match; + } + return requireWireToolName(undefined, canonicalName, "google"); + }, + streamingToolCall: googleToolCall, +}; + +const commandCodeDriver: ToolWireDriver = { + observeOutbound: observeHttpOutbound, + extractWireToolName(body, canonicalName) { + const parsed = JSON.parse(body) as { params?: { tools?: Array<{ name?: string }> } }; + const match = parsed.params?.tools?.find(tool => tool.name?.includes(canonicalName))?.name; + return requireWireToolName(match, canonicalName, "command-code"); + }, + streamingToolCall: commandCodeToolCall, +}; + +const kiroDriver: ToolWireDriver = { + observeOutbound: observeHttpOutbound, + extractWireToolName(body, canonicalName) { + const parsed = JSON.parse(body) as { + conversationState?: { + currentMessage?: { + userInputMessage?: { + userInputMessageContext?: { + tools?: Array<{ toolSpecification?: { name?: string } }>; + }; + }; + }; + }; + }; + const tools = parsed.conversationState?.currentMessage?.userInputMessage?.userInputMessageContext?.tools ?? []; + const match = tools.find(tool => tool.toolSpecification?.name?.includes(canonicalName))?.toolSpecification?.name; + return requireWireToolName(match, canonicalName, "kiro"); + }, + streamingToolCall: kiroToolCall, +}; + +const responsesDriver: ToolWireDriver = { + observeOutbound: observeHttpOutbound, + extractWireToolName(body, canonicalName) { + const parsed = JSON.parse(body) as { tools?: Array<{ name?: string }> }; + const match = parsed.tools?.find(tool => tool.name?.includes(canonicalName))?.name; + return requireWireToolName(match, canonicalName, "openai-responses"); + }, +}; + +export const TOOL_WIRE_DRIVERS = { + "openai-chat": openAiChatDriver, + anthropic: anthropicDriver, + google: googleDriver, + "command-code": commandCodeDriver, + kiro: kiroDriver, + "openai-responses": responsesDriver, + cursor: { + async observeOutbound(_adapter, parsed) { + const request = createCursorRequest(parsed); + const prepared = prepareCursorRunRequest(request); + try { + const message = fromBinary(AgentClientMessageSchema, prepared.bytes); + if (message.message.case !== "runRequest") { + throw new Error(`Cursor conformance expected runRequest, got ${message.message.case || "empty"}`); + } + const runRequest = message.message.value; + const tools = runRequest.mcpTools?.mcpTools ?? []; + const continuationToolCalls: Array<{ name: string; arguments: Record<string, unknown> }> = []; + for (const turnId of runRequest.conversationState?.turns ?? []) { + const turn = fromBinary( + ConversationTurnStructureSchema, + cursorBlobData(turnId, prepared.blobRequestScope), + ); + if (turn.turn.case !== "agentConversationTurn") continue; + for (const stepId of turn.turn.value.steps) { + const step = fromBinary( + ConversationStepSchema, + cursorBlobData(stepId, prepared.blobRequestScope), + ); + if (step.message.case !== "toolCall") continue; + const tool = step.message.value.tool; + if (tool.case !== "mcpToolCall") continue; + const args = tool.value.args; + continuationToolCalls.push({ + name: args?.toolName || args?.name || "", + arguments: decodeCursorArgsMap(args?.args), + }); + } + } + return JSON.stringify({ + tools: tools.map(tool => ({ + name: tool.toolName || tool.name, + description: tool.description, + })), + continuationToolCalls, + }); + } finally { + releaseCursorBlobRequestScope(prepared.blobRequestScope); + } + }, + }, +} satisfies Record<AdapterWire, ToolWireDriver>; diff --git a/tests/helpers/startup-health.ts b/tests/helpers/startup-health.ts new file mode 100644 index 0000000000..f7cb47d8c4 --- /dev/null +++ b/tests/helpers/startup-health.ts @@ -0,0 +1,33 @@ +import type { StartupHealth } from "../../src/codex/autostart-health"; + +export function startupHealthFixture(overrides: Partial<StartupHealth> = {}): StartupHealth { + return { + status: "native", + routingKind: "native", + routingInjected: false, + localRoutingDependency: false, + autostartEnabled: false, + rebootSafe: true, + protection: "none", + serviceInstalled: false, + serviceViable: false, + serviceEnabled: false, + serviceRunning: false, + serviceStale: false, + serviceConflict: false, + shimInstalled: false, + shimHealthy: false, + shimCoverage: "none", + serviceSupported: true, + platform: process.platform, + diagnosticStale: false, + recommendedCommand: null, + commands: { + installService: "ocx service install", + repairService: "ocx service repair", + installShim: "ocx codex-shim install", + restoreNative: "ocx restore", + }, + ...overrides, + }; +} diff --git a/tests/lab-activation.test.ts b/tests/lab-activation.test.ts index ef3c6fed4f..d056af9c46 100644 --- a/tests/lab-activation.test.ts +++ b/tests/lab-activation.test.ts @@ -13,8 +13,13 @@ import { resolveCompatibilityEvidenceProvider, resetCompatibilityEvidenceProviderForTests, } from "../src/routing/compatibility/provider-slot"; +import { defaultLabAutomationPolicyV1 } from "../src/lab/automation/policy"; import { isLabAutomationSchedulerRunning, stopLabAutomationScheduler } from "../src/lab/automation/orchestrator"; import { runOptionalShutdownHooks, resetOptionalShutdownHooksForTests } from "../src/lib/optional-shutdown-hooks"; +import { + acquireServerResourceOwner, + resetServerResourceOwnershipForTests, +} from "../src/lib/server-resource-ownership"; import { hasPassiveRouteLinker, resetPassiveRouteLinkerForTests } from "../src/server/passive-route-linker"; import type { OcxConfig } from "../src/types"; @@ -23,7 +28,23 @@ function scratch(): string { mkdirSync(join(dir, "lab"), { recursive: true }); return dir; } -const withProfile = { providers: {}, routingProfiles: { p: { candidates: [] } } } as unknown as OcxConfig; + +function writeEnabledAutomationConfig(dir: string): void { + writeFileSync(join(dir, "lab", "automation-config.json"), JSON.stringify({ + schemaVersion: 1, + policy: { + ...defaultLabAutomationPolicyV1(), + enabled: true, + }, + routes: { schemaVersion: 1, routes: [] }, + })); +} + +function profileConfig(): OcxConfig { + return { providers: {}, routingProfiles: { p: { candidates: [] } } } as unknown as OcxConfig; +} + +const withProfile = profileConfig(); const bare = { providers: {} } as unknown as OcxConfig; describe("lab activation gate", () => { @@ -31,6 +52,7 @@ describe("lab activation gate", () => { // otherwise leak into the bare-install assertion below. beforeEach(() => { resetLabActivationForTests(); + resetServerResourceOwnershipForTests(); resetCompatibilityEvidenceProviderForTests(); resetPassiveRouteLinkerForTests(); }); @@ -79,6 +101,40 @@ describe("lab activation gate", () => { expect(isLabActivated(dir)).toBe(true); }); + test("same-root successor keeps automation after predecessor owner release", () => { + const dir = scratch(); + writeEnabledAutomationConfig(dir); + const firstOwner = acquireServerResourceOwner(); + activateLab(profileConfig(), dir); + expect(isLabAutomationSchedulerRunning(dir)).toBe(true); + + const secondOwner = acquireServerResourceOwner(); + activateLab(profileConfig(), dir); + firstOwner.release(); + expect(isLabAutomationSchedulerRunning(dir)).toBe(true); + + secondOwner.release(); + expect(isLabAutomationSchedulerRunning(dir)).toBe(false); + }); + + test("same-process restart reacquires automation after its prior owner ended", () => { + const dir = scratch(); + writeEnabledAutomationConfig(dir); + const config = profileConfig(); + + const firstOwner = acquireServerResourceOwner(); + activateLab(config, dir); + expect(isLabAutomationSchedulerRunning(dir)).toBe(true); + firstOwner.release(); + expect(isLabAutomationSchedulerRunning(dir)).toBe(false); + + const secondOwner = acquireServerResourceOwner(); + activateLab(config, dir); + expect(isLabAutomationSchedulerRunning(dir)).toBe(true); + secondOwner.release(); + expect(isLabAutomationSchedulerRunning(dir)).toBe(false); + }); + // Ordering trap: automation-only activation must not permanently satisfy a later // profile-driven one. Safe today only because activation is all-or-nothing; this test // fails the moment a registration becomes conditional on the activation reason. @@ -100,6 +156,7 @@ describe("automation detection reads the current authority", () => { // otherwise leak into the bare-install assertion below. beforeEach(() => { resetLabActivationForTests(); + resetServerResourceOwnershipForTests(); resetCompatibilityEvidenceProviderForTests(); resetPassiveRouteLinkerForTests(); }); @@ -139,7 +196,11 @@ describe("automation detection reads the current authority", () => { }); describe("failed scheduler start leaves nothing dangling", () => { - beforeEach(() => { resetLabActivationForTests(); resetOptionalShutdownHooksForTests(); }); + beforeEach(() => { + resetLabActivationForTests(); + resetServerResourceOwnershipForTests(); + resetOptionalShutdownHooksForTests(); + }); // startLabAutomationScheduler registers its shutdown hook BEFORE it can throw, so a // failed start leaves a hook with no timer behind it. That must be harmless: no running diff --git a/tests/lab-community-evidence.test.ts b/tests/lab-community-evidence.test.ts new file mode 100644 index 0000000000..2e72aa5618 --- /dev/null +++ b/tests/lab-community-evidence.test.ts @@ -0,0 +1,224 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { existsSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + LAB_EVENT_SCHEMA_VERSION, + LAB_PRODUCER, + assignEventId, + labLedgerPath, + labSqlitePath, + purgeSensitiveEvidence, + subjectIdForSubject, + type ObservationEvent, + type ProtocolSubjectV1, +} from "../src/lab"; +import { labCommunityDir } from "../src/lab/paths"; +import { + buildPublicEvidenceBundle, + createPublicEvidenceRevocation, + getOrCreatePublicPublisher, + importCommunityEvidenceBundle, + importCommunityEvidenceRevocation, + listCommunityEvidence, + projectPublicEvidence, + publicEvidenceId, + signPublicEvidenceBundle, + signPublicPublisherDigest, + verifyPublicEvidenceRevocation, + writePublicEvidenceBundle, +} from "../src/lab/public"; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix = "ocx-cl10-community-"): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function hex(seed: string): string { + return Bun.CryptoHasher.hash("sha256", seed, "hex"); +} + +function assertionsForScenario(scenarioId: string) { + const ids = scenarioId === "responses-core.protocol.sse-framing" + ? ["events", "text", "terminal"] + : ["method", "message", "temperature"]; + return ids.map((id) => ({ id, operator: "equals", required: true, passed: true })); +} + +function protocolObservation(scenarioId = "responses-core.protocol.request-shape"): ObservationEvent { + const subject: ProtocolSubjectV1 = { + subjectSchemaVersion: 1, + subjectKind: "protocol", + opencodexCompatibilityVersion: "2.13.0", + effectiveAdapter: "openai-chat", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + behaviorFingerprint: hex("PRIVATE-community-behavior"), + }; + return assignEventId({ + schemaVersion: LAB_EVENT_SCHEMA_VERSION, + eventKind: "observation" as const, + recordedAt: Date.UTC(2026, 7, 12, 14, 37, 48), + producer: LAB_PRODUCER, + producerVersion: "2.13.0", + evidenceLayer: "protocol_conformance" as const, + scenarioId, + scenarioVersion: "1.0.0", + scenarioManifestDigest: hex("scenario"), + suiteId: "responses-core", + suiteVersion: "1.0.0", + suiteManifestDigest: hex("suite"), + fixtureDigests: [hex("fixture")], + subject, + subjectId: subjectIdForSubject(subject), + startedAt: Date.UTC(2026, 7, 12, 14, 37, 40), + completedAt: Date.UTC(2026, 7, 12, 14, 37, 41), + executionMode: "fixture" as const, + attempt: 1, + limits: { totalTimeoutMs: 1000 }, + outcome: "pass" as const, + assertions: assertionsForScenario(scenarioId), + environment: {}, + artifactRefs: [], + }) as ObservationEvent; +} + +function signedBundle(config: string, scenarioId?: string) { + const projected = projectPublicEvidence({ + createdDayUtc: "2026-08-12", + records: [{ observation: protocolObservation(scenarioId), verdict: "VERIFIED" }], + }); + return signPublicEvidenceBundle({ + records: projected.bundle.records, + artifacts: projected.bundle.artifacts, + createdDayUtc: projected.bundle.createdDayUtc, + configDir: config, + }); +} + +function signedUnreviewedScenarioBundle(config: string) { + const projected = projectPublicEvidence({ + records: [{ observation: protocolObservation(), verdict: "VERIFIED" }], + }); + const baseRecord = projected.bundle.records[0]; + if (!baseRecord) throw new Error("expected reviewed source record"); + const { recordId: _recordId, ...baseFields } = baseRecord; + const withoutRecordId = { ...baseFields, scenarioId: "private.unknown.scenario" }; + const record = { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId }; + const handle = getOrCreatePublicPublisher(config); + const unsigned = buildPublicEvidenceBundle({ + records: [record], + artifacts: [], + createdDayUtc: projected.bundle.createdDayUtc, + publisher: handle.publisher, + }); + return { + ...unsigned, + signature: { + algorithm: "ed25519" as const, + signedDigest: unsigned.bundleDigest, + signature: signPublicPublisherDigest(handle, unsigned.bundleDigest), + }, + }; +} + +describe("CL-10 community quarantine", () => { + test("imports valid signed evidence without touching canonical Lab authority", () => { + const publisherDir = configDir("ocx-cl10-publisher-"); + const consumerDir = configDir("ocx-cl10-consumer-"); + const bundle = signedBundle(publisherDir); + const imported = importCommunityEvidenceBundle(bundle, consumerDir); + expect(imported).toMatchObject({ created: true, status: "cryptographically_valid", bundleId: bundle.bundleId }); + expect(existsSync(labLedgerPath(consumerDir))).toBe(false); + expect(existsSync(labSqlitePath(consumerDir))).toBe(false); + expect(listCommunityEvidence(consumerDir)).toEqual([expect.objectContaining({ + bundleId: bundle.bundleId, + status: "cryptographically_valid", + activeRecordCount: 1, + revokedRecordCount: 0, + })]); + expect(importCommunityEvidenceBundle(bundle, consumerDir).created).toBe(false); + }); + + test("rejects cryptographically valid but unknown scenario authority", () => { + const publisherDir = configDir("ocx-cl10-publisher-"); + const consumerDir = configDir("ocx-cl10-consumer-"); + const bundle = signedUnreviewedScenarioBundle(publisherDir); + expect(() => importCommunityEvidenceBundle(bundle, consumerDir)).toThrow(/authority/i); + expect(listCommunityEvidence(consumerDir)).toEqual([]); + }); + + test("same-key revocation is verified, idempotent, and removes records from default community context", () => { + const publisherDir = configDir("ocx-cl10-publisher-"); + const consumerDir = configDir("ocx-cl10-consumer-"); + const bundle = signedBundle(publisherDir); + importCommunityEvidenceBundle(bundle, consumerDir); + + const revocation = createPublicEvidenceRevocation({ + configDir: publisherDir, + targetBundle: bundle, + issuedDayUtc: "2026-08-12", + reason: "evidence_invalidated", + targets: [{ kind: "record", id: bundle.records[0]!.recordId }], + }); + expect(verifyPublicEvidenceRevocation(revocation, bundle).status).toBe("cryptographically_valid"); + expect(importCommunityEvidenceRevocation(revocation, consumerDir).created).toBe(true); + expect(importCommunityEvidenceRevocation(revocation, consumerDir).created).toBe(false); + expect(listCommunityEvidence(consumerDir)[0]).toMatchObject({ activeRecordCount: 0, revokedRecordCount: 1 }); + }); + + test("rejects cross-key revocation and conflicting same-id stored bytes", () => { + const publisherDir = configDir("ocx-cl10-publisher-"); + const otherDir = configDir("ocx-cl10-other-"); + const consumerDir = configDir("ocx-cl10-consumer-"); + const bundle = signedBundle(publisherDir); + importCommunityEvidenceBundle(bundle, consumerDir); + expect(() => createPublicEvidenceRevocation({ + configDir: otherDir, + targetBundle: bundle, + issuedDayUtc: "2026-08-12", + reason: "publisher_retracted", + targets: [{ kind: "bundle", id: bundle.bundleId }], + })).toThrow(/publisher/i); + + const revocation = createPublicEvidenceRevocation({ + configDir: publisherDir, + targetBundle: bundle, + issuedDayUtc: "2026-08-12", + reason: "publisher_retracted", + targets: [{ kind: "bundle", id: bundle.bundleId }], + }); + const conflictPath = join(labCommunityDir(consumerDir), `revocation-${revocation.revocationId}.json`); + writeFileSync(conflictPath, JSON.stringify({ conflicting: true }), { mode: 0o600 }); + expect(() => importCommunityEvidenceRevocation(revocation, consumerDir)).toThrow(/identity.*different bytes|conflict/i); + }); + + test("sensitive export purge removes local exports and local community copies but preserves third-party bundles", () => { + const consumerDir = configDir("ocx-cl10-consumer-"); + const thirdPartyDir = configDir("ocx-cl10-third-party-"); + const localBundle = signedBundle(consumerDir); + const localStored = writePublicEvidenceBundle(localBundle, consumerDir); + importCommunityEvidenceBundle(localBundle, consumerDir); + + const thirdPartyBundle = signedBundle(thirdPartyDir, "responses-core.protocol.sse-framing"); + importCommunityEvidenceBundle(thirdPartyBundle, consumerDir); + expect(listCommunityEvidence(consumerDir)).toHaveLength(2); + + purgeSensitiveEvidence({ + configDir: consumerDir, + targetArtifactDigests: [hex("sensitive-purge-target")], + purgeActions: ["export"], + recordedAt: Date.UTC(2026, 7, 12, 18, 0, 0), + }); + + expect(existsSync(localStored)).toBe(false); + expect(listCommunityEvidence(consumerDir).map((row) => row.bundleId)).toEqual([thirdPartyBundle.bundleId]); + }); +}); \ No newline at end of file diff --git a/tests/lab-community-filename-contract.test.ts b/tests/lab-community-filename-contract.test.ts new file mode 100644 index 0000000000..594d654b20 --- /dev/null +++ b/tests/lab-community-filename-contract.test.ts @@ -0,0 +1,123 @@ +import { afterEach, expect, test } from "bun:test"; +import { + existsSync, + mkdtempSync, + readdirSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { basename, join } from "node:path"; +import { labPublicOriginDir } from "../src/lab/paths"; +import { + communityBundleFileName, + communityRevocationFileName, +} from "../src/lab/public/community-files"; +import { + createPublicEvidenceRevocation, + importCommunityEvidenceBundle, + importCommunityEvidenceRevocation, + purgeLocalPublicEvidenceCopies, + publicEvidenceId, + recordLocalPublicOrigin, + signPublicEvidenceBundle, + type PublicEvidenceRecordV1, +} from "../src/lab/public"; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(): string { + const root = mkdtempSync(join(tmpdir(), "ocx-cl10-community-filenames-")); + roots.push(root); + return root; +} + +function hex(seed: string): string { + return Bun.CryptoHasher.hash("sha256", seed, "hex"); +} + +function fixedRecord(): PublicEvidenceRecordV1 { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const withoutRecordId = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: "2026-08-12", + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId }; +} + +test("writer, origin retention, and purge share the community filename contract", () => { + const home = configDir(); + const bundle = signPublicEvidenceBundle({ + records: [fixedRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + configDir: home, + }); + const importedBundle = importCommunityEvidenceBundle(bundle, home); + expect(basename(importedBundle.path)).toBe( + communityBundleFileName(bundle.publisher.keyId, bundle.bundleId), + ); + + const revocation = createPublicEvidenceRevocation({ + configDir: home, + targetBundle: bundle, + issuedDayUtc: "2026-08-13", + targets: [{ kind: "bundle", id: bundle.bundleId }], + reason: "publisher_retracted", + }); + const importedRevocation = importCommunityEvidenceRevocation(revocation, home); + expect(basename(importedRevocation.path)).toBe( + communityRevocationFileName(revocation.revocationId), + ); + + recordLocalPublicOrigin({ + publisherKeyId: bundle.publisher.keyId, + bundleId: bundle.bundleId, + }, home); + + const originDir = labPublicOriginDir(home); + for (let index = 0; index < 1023; index += 1) { + writeFileSync( + join(originDir, `origin-${hex(`stale-publisher-${index}`)}-${hex(`stale-bundle-${index}`)}.json`), + "{}", + { mode: 0o600 }, + ); + } + recordLocalPublicOrigin({ + publisherKeyId: hex("new-publisher"), + bundleId: hex("new-bundle"), + }, home); + + expect(readdirSync(originDir)).toContain( + `origin-${bundle.publisher.keyId}-${bundle.bundleId}.json`, + ); + + const purged = purgeLocalPublicEvidenceCopies(home); + expect(purged.deletedCommunityBundles).toBe(1); + expect(purged.deletedCommunityRevocations).toBe(1); + expect(existsSync(importedBundle.path)).toBe(false); + expect(existsSync(importedRevocation.path)).toBe(false); +}); diff --git a/tests/lab-community-mutation-lock.test.ts b/tests/lab-community-mutation-lock.test.ts new file mode 100644 index 0000000000..68ef4e7b6c --- /dev/null +++ b/tests/lab-community-mutation-lock.test.ts @@ -0,0 +1,125 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { existsSync, mkdirSync, mkdtempSync, rmSync, utimesSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ensureLabDirs, labCommunityDir } from "../src/lab/paths"; +import { listCommunityEvidence } from "../src/lab/public/community"; +import { + publicEvidenceMutationLockIsReclaimableForTests, + publicEvidenceTryReclaimMutationLockForTests, +} from "../src/lab/public/mutation-lock"; +import { PublicEvidenceValidationError } from "../src/lab/public/validate"; + +const roots: string[] = []; + +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(): string { + const root = mkdtempSync(join(tmpdir(), "ocx-community-lock-")); + roots.push(root); + ensureLabDirs(root); + return root; +} + +function createLiveOwnerLock(config: string): string { + const lockPath = join(labCommunityDir(config), ".mutation-lock"); + mkdirSync(lockPath, { mode: 0o700 }); + writeFileSync( + join(lockPath, "owner.json"), + JSON.stringify({ + pid: process.pid, + token: "00000000-0000-4000-8000-000000000000", + createdAt: Date.now(), + }), + { encoding: "utf8", mode: 0o600 }, + ); + return lockPath; +} + +describe("community mutation lock", () => { + test("recovers an ancient incomplete lock before reading committed cache state", () => { + const config = configDir(); + const lockPath = join(labCommunityDir(config), ".mutation-lock"); + mkdirSync(lockPath, { mode: 0o700 }); + const stale = new Date(Date.now() - (25 * 60 * 60 * 1000)); + utimesSync(lockPath, stale, stale); + + expect(listCommunityEvidence(config)).toEqual([]); + expect(existsSync(lockPath)).toBe(false); + }); + + test("does not reclaim an old lock while its recorded owner process is alive", () => { + const config = configDir(); + const lockPath = join(labCommunityDir(config), ".mutation-lock"); + mkdirSync(lockPath, { mode: 0o700 }); + writeFileSync( + join(lockPath, "owner.json"), + JSON.stringify({ + pid: process.pid, + token: "00000000-0000-4000-8000-000000000000", + createdAt: Date.now() - (25 * 60 * 60 * 1000), + }), + { encoding: "utf8", mode: 0o600 }, + ); + + expect(publicEvidenceMutationLockIsReclaimableForTests(config)).toBe(false); + expect(existsSync(lockPath)).toBe(true); + }); + + test("fails fast when a live owner holds the mutation lock", () => { + const config = configDir(); + const lockPath = createLiveOwnerLock(config); + const startedAt = performance.now(); + let failure: unknown; + + try { + listCommunityEvidence(config); + } catch (error) { + failure = error; + } + + const elapsedMs = performance.now() - startedAt; + expect(failure).toBeInstanceOf(PublicEvidenceValidationError); + expect((failure as PublicEvidenceValidationError).code).toBe("community_cache_busy"); + expect(elapsedMs).toBeLessThan(500); + expect(existsSync(lockPath)).toBe(true); + }); + + test("a competing reclaim claim prevents a second stale reclaimer from deleting the lock", () => { + const config = configDir(); + const lockPath = join(labCommunityDir(config), ".mutation-lock"); + mkdirSync(lockPath, { mode: 0o700 }); + const stale = new Date(Date.now() - (25 * 60 * 60 * 1000)); + utimesSync(lockPath, stale, stale); + writeFileSync( + join(lockPath, ".reclaim.json"), + JSON.stringify({ token: "00000000-0000-4000-8000-000000000000" }), + { encoding: "utf8", mode: 0o600 }, + ); + + expect(publicEvidenceTryReclaimMutationLockForTests(config)).toBe(false); + expect(existsSync(lockPath)).toBe(true); + }); + + test("cleans detached lock quarantine before scanning community quota state", () => { + const config = configDir(); + const quarantinePath = join( + labCommunityDir(config), + ".mutation-lock-release-123-00000000-0000-4000-8000-000000000000", + ); + mkdirSync(quarantinePath, { mode: 0o700 }); + writeFileSync(join(quarantinePath, "owner.json"), "stale", { mode: 0o600 }); + + expect(listCommunityEvidence(config)).toEqual([]); + expect(existsSync(quarantinePath)).toBe(false); + }); + + test("fails closed when the lock path is not a directory", () => { + const config = configDir(); + writeFileSync(join(labCommunityDir(config), ".mutation-lock"), "unsafe", "utf8"); + + expect(() => listCommunityEvidence(config)).toThrow(/mutation lock is not a directory/i); + }); +}); diff --git a/tests/lab-community-publisher-continuity.test.ts b/tests/lab-community-publisher-continuity.test.ts new file mode 100644 index 0000000000..5652379e0f --- /dev/null +++ b/tests/lab-community-publisher-continuity.test.ts @@ -0,0 +1,120 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + LAB_EVENT_SCHEMA_VERSION, + LAB_PRODUCER, + assignEventId, + subjectIdForSubject, + type ObservationEvent, + type ProtocolSubjectV1, +} from "../src/lab"; +import { + createPublicEvidenceRevocation, + importCommunityEvidenceBundle, + importCommunityEvidenceRevocation, + listCommunityEvidence, + projectPublicEvidence, + signPublicEvidenceBundle, +} from "../src/lab/public"; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function hex(seed: string): string { + return Bun.CryptoHasher.hash("sha256", seed, "hex"); +} + +function observation(): ObservationEvent { + const subject: ProtocolSubjectV1 = { + subjectSchemaVersion: 1, + subjectKind: "protocol", + opencodexCompatibilityVersion: "2.13.0", + effectiveAdapter: "openai-chat", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + behaviorFingerprint: hex("PRIVATE-publisher-continuity"), + }; + return assignEventId({ + schemaVersion: LAB_EVENT_SCHEMA_VERSION, + eventKind: "observation" as const, + recordedAt: Date.UTC(2026, 7, 12, 14, 37, 48), + producer: LAB_PRODUCER, + producerVersion: "2.13.0", + evidenceLayer: "protocol_conformance" as const, + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + scenarioManifestDigest: hex("scenario"), + suiteId: "responses-core", + suiteVersion: "1.0.0", + suiteManifestDigest: hex("suite"), + fixtureDigests: [hex("fixture")], + subject, + subjectId: subjectIdForSubject(subject), + startedAt: Date.UTC(2026, 7, 12, 14, 37, 40), + completedAt: Date.UTC(2026, 7, 12, 14, 37, 41), + executionMode: "fixture" as const, + attempt: 1, + limits: { totalTimeoutMs: 1000 }, + outcome: "pass" as const, + assertions: [ + { id: "method", operator: "equals", required: true, passed: true }, + { id: "message", operator: "equals", required: true, passed: true }, + { id: "temperature", operator: "equals", required: true, passed: true }, + ], + environment: {}, + artifactRefs: [], + }) as ObservationEvent; +} + +function projectedBundle() { + return projectPublicEvidence({ + createdDayUtc: "2026-08-12", + records: [{ observation: observation(), verdict: "VERIFIED" }], + }).bundle; +} + +describe("CL-10 publisher continuity", () => { + test("same content from two publishers coexists and revokes independently", () => { + const publisherA = configDir("ocx-cl10-publisher-a-"); + const publisherB = configDir("ocx-cl10-publisher-b-"); + const consumer = configDir("ocx-cl10-consumer-"); + const unsigned = projectedBundle(); + const bundleA = signPublicEvidenceBundle({ ...unsigned, configDir: publisherA }); + const bundleB = signPublicEvidenceBundle({ ...unsigned, configDir: publisherB }); + + expect(bundleA.bundleId).not.toBe(bundleB.bundleId); + expect(bundleA.publisher.keyId).not.toBe(bundleB.publisher.keyId); + expect(importCommunityEvidenceBundle(bundleA, consumer).created).toBe(true); + expect(importCommunityEvidenceBundle(bundleB, consumer).created).toBe(true); + + let summaries = listCommunityEvidence(consumer); + expect(summaries).toHaveLength(2); + expect(new Set(summaries.map((row) => row.publisherKeyId)).size).toBe(2); + + const revocationA = createPublicEvidenceRevocation({ + configDir: publisherA, + targetBundle: bundleA, + issuedDayUtc: "2026-08-12", + reason: "publisher_retracted", + targets: [{ kind: "bundle", id: bundleA.bundleId }], + }); + expect(importCommunityEvidenceRevocation(revocationA, consumer).created).toBe(true); + + summaries = listCommunityEvidence(consumer); + const rowA = summaries.find((row) => row.publisherKeyId === bundleA.publisher.keyId)!; + const rowB = summaries.find((row) => row.publisherKeyId === bundleB.publisher.keyId)!; + expect(rowA).toMatchObject({ activeRecordCount: 0, revokedRecordCount: 1 }); + expect(rowB).toMatchObject({ activeRecordCount: 1, revokedRecordCount: 0 }); + }); +}); \ No newline at end of file diff --git a/tests/lab-conformance-runner-failures.test.ts b/tests/lab-conformance-runner-failures.test.ts new file mode 100644 index 0000000000..47fc707bd9 --- /dev/null +++ b/tests/lab-conformance-runner-failures.test.ts @@ -0,0 +1,53 @@ +import { describe, expect, test } from "bun:test"; +import { runScenario } from "../src/lab/conformance/executor"; +import { NEGATIVE_CONTROL_FIXTURES } from "../src/lab/conformance/negative-controls"; +import { runNegativeControls } from "../src/lab/conformance/runner"; + +function unexpectedNegativeControlResults( + results: Awaited<ReturnType<typeof runNegativeControls>>["results"], +): string[] { + return results + .filter((result) => ( + result.passed + || result.classification !== "protocol_failure" + || result.secondaryCode !== "deterministic_assertion" + )) + .map((result) => ( + `${result.scenarioId}: ${result.classification}/${result.secondaryCode ?? "none"}` + + (result.diagnostics.length > 0 ? ` ${result.diagnostics.join(";")}` : "") + )); +} + +describe("CL-01 negative-control failure accounting", () => { + test("does not count harness failures as rejected negative controls", async () => { + let injected = false; + const summary = await runNegativeControls(async (scenario) => { + const result = await runScenario(scenario); + if (injected) return result; + injected = true; + return { + ...result, + passed: false, + classification: "harness_failure", + secondaryCode: "execution_error", + assertionResults: [], + diagnostics: ["synthetic harness failure"], + }; + }); + + expect(summary.total).toBe(NEGATIVE_CONTROL_FIXTURES.length); + expect(summary.rejected).toBe(summary.total - 1); + expect(summary.passed).toBe(summary.rejected); + expect(summary.failed).toBe(1); + expect(summary.results.filter((result) => result.classification === "harness_failure")).toHaveLength(1); + }, 120000); + + test("counts deterministic protocol failures as rejected negative controls", async () => { + const summary = await runNegativeControls(); + + expect(summary.total).toBe(NEGATIVE_CONTROL_FIXTURES.length); + expect(unexpectedNegativeControlResults(summary.results)).toEqual([]); + expect(summary.rejected).toBe(summary.total); + expect(summary.failed).toBe(0); + }, 120000); +}); diff --git a/tests/lab-fabric-outcome-validation.test.ts b/tests/lab-fabric-outcome-validation.test.ts new file mode 100644 index 0000000000..d5d0ee0bb7 --- /dev/null +++ b/tests/lab-fabric-outcome-validation.test.ts @@ -0,0 +1,126 @@ +import { describe, expect, test } from "bun:test"; +import { + assertFabricOutcomeV1, + buildTaskSubjectV1, + FABRIC_LIMITS, + FABRIC_VERIFIER_ID, + subjectIdForSubject, + type FabricTaskOutcomeV1, + type RouteSubjectV1, +} from "../src/lab"; +import { FabricTaskError } from "../src/lab/fabric/types"; + +function routeSubject(): RouteSubjectV1 { + return { + subjectSchemaVersion: 1, + subjectKind: "route", + providerId: "provider-a", + providerInstanceFingerprint: "a".repeat(64), + clientModelId: "model-a", + upstreamModelId: "model-a", + effectiveAdapter: "openai-responses", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-responses", + surface: "responses-http", + opencodexCompatibilityVersion: "b".repeat(64), + behaviorFingerprint: "c".repeat(64), + endpointFingerprint: "d".repeat(64), + dependencies: [], + }; +} + +function validOutcome(): FabricTaskOutcomeV1 { + const taskSubject = buildTaskSubjectV1({ routeSubject: routeSubject() }); + return { + schemaVersion: 1, + taskClassId: taskSubject.taskClassId, + taskClassVersion: taskSubject.taskClassVersion, + routeSubject: taskSubject.routeSubject, + taskSubject, + subjectId: subjectIdForSubject(taskSubject), + taskFixtureDigest: taskSubject.taskFixtureDigest, + verifierManifestDigest: taskSubject.verifierManifestDigest, + fabricCompatibilityVersion: taskSubject.fabricCompatibilityVersion, + sandboxProfileDigest: taskSubject.sandboxProfileDigest, + startedAt: 100, + completedAt: 200, + limits: { ...FABRIC_LIMITS }, + usage: { + inputBytes: 7, + outputBytes: 6, + patchOperations: 1, + filesTouched: 1, + artifactBytes: 0, + elapsedMs: 100, + inactiveMs: 0, + }, + outcome: "pass", + verifier: { + verifierId: FABRIC_VERIFIER_ID, + manifestDigest: taskSubject.verifierManifestDigest, + passed: true, + pathSummaries: [], + }, + artifactDigests: [], + }; +} + +function changedDigest(value: string): string { + return `${value[0] === "0" ? "1" : "0"}${value.slice(1)}`; +} + +describe("CL-07 fabric outcome validation", () => { + test("accepts a canonical outcome", () => { + const outcome = validOutcome(); + expect(assertFabricOutcomeV1(outcome)).toBe(outcome); + }); + + test("rejects undeclared nested producer fields", () => { + const outcome = validOutcome(); + const malformed = [ + { ...outcome, usage: { ...outcome.usage, credential: "secret" } }, + { ...outcome, limits: { ...outcome.limits, unexpectedLimit: 1 } }, + { ...outcome, taskSubject: { ...outcome.taskSubject, credential: "secret" } }, + { + ...outcome, + taskSubject: { + ...outcome.taskSubject, + routeSubject: { ...outcome.taskSubject.routeSubject, credential: "secret" }, + }, + }, + { ...outcome, routeSubject: { ...outcome.routeSubject, credential: "secret" } }, + ]; + + for (const candidate of malformed) { + expect(() => assertFabricOutcomeV1(candidate)).toThrow(FabricTaskError); + } + }); + + test("rejects negative or reversed execution timestamps", () => { + const outcome = validOutcome(); + expect(() => assertFabricOutcomeV1({ ...outcome, startedAt: -1 })).toThrow(FabricTaskError); + expect(() => assertFabricOutcomeV1({ ...outcome, startedAt: 201 })).toThrow(FabricTaskError); + expect(() => assertFabricOutcomeV1({ ...outcome, startedAt: 100.5 })).toThrow(FabricTaskError); + expect(() => assertFabricOutcomeV1({ ...outcome, completedAt: 200.5 })).toThrow(FabricTaskError); + }); + + test("rejects contradictory task identity fields", () => { + const outcome = validOutcome(); + const malformed = [ + { ...outcome, subjectId: changedDigest(outcome.subjectId) }, + { ...outcome, taskClassId: `${outcome.taskClassId}-other` }, + { ...outcome, taskFixtureDigest: changedDigest(outcome.taskFixtureDigest) }, + { + ...outcome, + verifier: { + ...outcome.verifier, + manifestDigest: changedDigest(outcome.verifier.manifestDigest), + }, + }, + ]; + + for (const candidate of malformed) { + expect(() => assertFabricOutcomeV1(candidate)).toThrow(FabricTaskError); + } + }); +}); diff --git a/tests/lab-ledger-mutation-lock.test.ts b/tests/lab-ledger-mutation-lock.test.ts new file mode 100644 index 0000000000..d659ca4d73 --- /dev/null +++ b/tests/lab-ledger-mutation-lock.test.ts @@ -0,0 +1,263 @@ +import { afterEach, expect, test } from "bun:test"; +import { existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { + appendLabEvent, + appendLabEventIfAbsent, + assignEventId, + createArtifactStore, + LAB_EVENT_SCHEMA_VERSION, + LAB_PRODUCER, + LAB_PRODUCER_VERSION, + persistConformanceResult, + purgeSensitiveEvidence, + replayLabLedger, + withLedgerMutation, +} from "../src/lab"; +import type { ArtifactStore } from "../src/lab/artifacts/store"; +import { discoverScenarios, loadCaseAuthority } from "../src/lab/conformance/manifest"; +import { resolveProtocolExecutionContext } from "../src/lab/conformance/executor"; +import type { CaseRecord } from "../src/lab/conformance/types"; +import type { InvalidationEvent } from "../src/lab/events/types"; + +const HOMES: string[] = []; + +function tempHome(): string { + const dir = join(tmpdir(), `ocx-lab-ledger-lock-${process.pid}-${Math.random().toString(16).slice(2)}`); + mkdirSync(dir, { recursive: true, mode: 0o700 }); + HOMES.push(dir); + return dir; +} + +afterEach(() => { + for (const dir of HOMES.splice(0)) { + try { + rmSync(dir, { recursive: true, force: true }); + } catch { + /* ignore */ + } + } +}); + +function hash(value: string): string { + return Bun.CryptoHasher.hash("sha256", value, "hex"); +} + +function invalidation(seed: string, recordedAt = 1_700_000_000_000): InvalidationEvent { + return assignEventId({ + schemaVersion: LAB_EVENT_SCHEMA_VERSION, + eventKind: "invalidation" as const, + recordedAt, + producer: LAB_PRODUCER, + producerVersion: LAB_PRODUCER_VERSION, + targetEventIds: [hash(`target:${seed}`)], + reason: "manual_correction" as const, + }) as InvalidationEvent; +} + +function syntheticPassResult(caseRecord: CaseRecord) { + return { + scenarioId: caseRecord.id, + suite: caseRecord.suite, + passed: true, + classification: "inconclusive" as const, + assertionResults: caseRecord.assertions.map((assertion) => ({ + id: assertion.id, + operator: assertion.operator, + required: assertion.required, + passed: true, + observedSummary: "ok", + })), + diagnostics: [], + executionContext: resolveProtocolExecutionContext(caseRecord), + startedAt: 999, + completedAt: 1000, + }; +} + +async function waitForPath(path: string): Promise<void> { + for (let attempt = 0; attempt < 500; attempt += 1) { + if (existsSync(path)) return; + await Bun.sleep(10); + } + throw new Error(`timed out waiting for child marker ${path}`); +} + +async function waitForChild(child: ReturnType<typeof Bun.spawn>): Promise<void> { + const result = await Promise.race([ + child.exited.then((exitCode) => ({ exitCode })), + Bun.sleep(5_000).then(() => null), + ]); + if (!result) { + child.kill(); + await child.exited; + throw new Error("timed out waiting for ledger-lock child"); + } + if (result.exitCode !== 0) { + const stderr = await new Response(child.stderr).text().catch(() => ""); + throw new Error(`ledger-lock child exited ${result.exitCode}: ${stderr}`); + } +} + +function spawnLiveLock( + ledgerPath: string, + readyPath: string, + releaseMarkerPath: string, +): ReturnType<typeof Bun.spawn> { + const lockPath = `${ledgerPath}.lock`; + const childSource = ` + import { mkdirSync, unlinkSync, writeFileSync } from "node:fs"; + import { dirname } from "node:path"; + mkdirSync(dirname(${JSON.stringify(lockPath)}), { recursive: true, mode: 0o700 }); + writeFileSync( + ${JSON.stringify(lockPath)}, + JSON.stringify({ pid: process.pid, createdAt: Date.now(), token: "live-holder" }), + { mode: 0o600 }, + ); + writeFileSync(${JSON.stringify(readyPath)}, "ready"); + Bun.sleepSync(250); + writeFileSync(${JSON.stringify(releaseMarkerPath)}, "releasing"); + unlinkSync(${JSON.stringify(lockPath)}); + `; + return Bun.spawn([process.execPath, "-e", childSource], { + cwd: join(import.meta.dir, ".."), + env: { ...process.env }, + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + }); +} + +test("appendLabEvent waits for the shared ledger mutation lock", async () => { + const home = tempHome(); + const ledgerPath = join(home, "lab", "compatibility.jsonl"); + const readyPath = join(home, "holder-ready"); + const releaseMarkerPath = join(home, "holder-releasing"); + const child = spawnLiveLock(ledgerPath, readyPath, releaseMarkerPath); + + try { + await waitForPath(readyPath); + const event = invalidation("append-lock"); + appendLabEvent(ledgerPath, event); + expect(existsSync(releaseMarkerPath)).toBe(true); + expect(replayLabLedger(ledgerPath).events.some((row) => row.eventId === event.eventId)).toBe(true); + } finally { + await waitForChild(child); + } +}); + +test("appendLabEventIfAbsent immediately recovers a lock owned by an exited process", async () => { + const home = tempHome(); + const ledgerPath = join(home, "lab", "compatibility.jsonl"); + const lockPath = `${ledgerPath}.lock`; + const readyPath = join(home, "dead-lock-written"); + mkdirSync(dirname(ledgerPath), { recursive: true, mode: 0o700 }); + + const childSource = ` + import { writeFileSync } from "node:fs"; + writeFileSync( + ${JSON.stringify(lockPath)}, + JSON.stringify({ pid: process.pid, createdAt: Date.now(), token: "dead-holder" }), + { mode: 0o600 }, + ); + writeFileSync(${JSON.stringify(readyPath)}, "ready"); + `; + const child = Bun.spawn([process.execPath, "-e", childSource], { + cwd: join(import.meta.dir, ".."), + env: { ...process.env }, + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + }); + await waitForPath(readyPath); + await waitForChild(child); + + const event = invalidation("dead-lock"); + expect(appendLabEventIfAbsent(ledgerPath, event)).toBe(true); + expect(existsSync(lockPath)).toBe(false); + expect(existsSync(`${lockPath}.recovery`)).toBe(false); + expect(replayLabLedger(ledgerPath).events.some((row) => row.eventId === event.eventId)).toBe(true); +}); + +test("withLedgerMutation rejects async callbacks and invalidates their context", async () => { + const home = tempHome(); + const ledgerPath = join(home, "lab", "compatibility.jsonl"); + const event = invalidation("async-callback"); + let continuation: Promise<void> | undefined; + let continuationError: unknown; + + expect(() => withLedgerMutation(ledgerPath, (mutation) => { + continuation = (async () => { + await Bun.sleep(1); + mutation.append(event); + })().catch((error) => { + continuationError = error; + }); + return continuation; + })).toThrow("ledger mutation callback must be synchronous"); + + await continuation; + expect(continuationError).toBeInstanceOf(Error); + expect((continuationError as Error).message).toContain("after its lock was released"); + expect(replayLabLedger(ledgerPath).events.some((row) => row.eventId === event.eventId)).toBe(false); +}); + +test("canonical persistence publishes artifacts while holding the ledger mutation lock", () => { + const home = tempHome(); + const ledgerPath = join(home, "lab", "compatibility.jsonl"); + const authority = loadCaseAuthority(); + const caseRecord = discoverScenarios(authority, ["responses-core"]).find( + (candidate) => candidate.id === "responses-core.protocol.request-shape", + )!; + const realStore = createArtifactStore(join(home, "lab", "artifacts")); + const guardedStore: ArtifactStore = { + ...realStore, + put(input) { + expect(existsSync(`${ledgerPath}.lock`)).toBe(true); + return realStore.put(input); + }, + }; + + try { + const { event } = persistConformanceResult( + syntheticPassResult(caseRecord), + caseRecord, + authority, + { configDir: home, recordedAt: 1000, artifactStore: guardedStore }, + ); + expect(replayLabLedger(ledgerPath).events.some((row) => row.eventId === event.eventId)).toBe(true); + expect(existsSync(`${ledgerPath}.lock`)).toBe(false); + } finally { + realStore.close(); + } +}); + +test("sensitive purge waits for the ledger mutation lock before rewriting", async () => { + const home = tempHome(); + const ledgerPath = join(home, "lab", "compatibility.jsonl"); + const event = invalidation("purge-lock"); + appendLabEvent(ledgerPath, event); + + const readyPath = join(home, "purge-holder-ready"); + const releaseMarkerPath = join(home, "purge-holder-releasing"); + const child = spawnLiveLock(ledgerPath, readyPath, releaseMarkerPath); + + try { + await waitForPath(readyPath); + purgeSensitiveEvidence({ + configDir: home, + targetEventIds: [event.eventId], + targetArtifactDigests: [], + purgeActions: ["ledger"], + recordedAt: 1_700_000_000_100, + }); + expect(existsSync(releaseMarkerPath)).toBe(true); + const replay = replayLabLedger(ledgerPath); + expect(replay.events.some((row) => row.eventId === event.eventId)).toBe(false); + expect(replay.events.some((row) => row.eventKind === "purge_tombstone")).toBe(true); + expect(existsSync(`${ledgerPath}.lock`)).toBe(false); + } finally { + await waitForChild(child); + } +}); diff --git a/tests/lab-live-pinned-timeouts.test.ts b/tests/lab-live-pinned-timeouts.test.ts new file mode 100644 index 0000000000..4e9a37bfb0 --- /dev/null +++ b/tests/lab-live-pinned-timeouts.test.ts @@ -0,0 +1,117 @@ +import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; +import { afterEach, describe, expect, test } from "bun:test"; +import { createLabAuthorizedPinnedSender } from "../src/lib/lab-live-pinned-sender"; +import type { LabCredentialLeaseV1, LabDestinationV1, LiveRunConfig } from "../src/lab/live/types"; + +const SERVERS: Server[] = []; + +afterEach(async () => { + for (const server of SERVERS.splice(0)) { + server.closeAllConnections?.(); + await new Promise<void>((resolve) => server.close(() => resolve())); + } +}); + +async function listen(handler: (req: IncomingMessage, res: ServerResponse) => void): Promise<number> { + return await new Promise<number>((resolve, reject) => { + const server = createServer(handler); + SERVERS.push(server); + const onError = (error: Error) => reject(error); + server.once("error", onError); + server.listen(0, "127.0.0.1", () => { + server.off("error", onError); + const address = server.address(); + if (!address || typeof address === "string") { + reject(new Error("loopback test server did not expose a TCP port")); + return; + } + resolve(address.port); + }); + }); +} + +const BASE_LIMITS: LiveRunConfig = { + totalTimeoutMs: 1_000, + connectTimeoutMs: 250, + firstByteTimeoutMs: 30, + inactivityTimeoutMs: 30, + maxRequests: 2, + maxInputBytes: 1024, + maxOutputBytes: 1024, + maxOutputTokens: 1024, + maxToolCalls: 8, + maxMemoryBytes: 64 * 1024 * 1024, + maxChildProcesses: 0, + maxArtifacts: 4, + perArtifactBytes: 64 * 1024, + aggregateArtifactBytes: 256 * 1024, +}; + +function destination(port: number): LabDestinationV1 { + return { + scheme: "http", + host: "lab-timeout.invalid", + port, + basePath: "", + sniHost: "lab-timeout.invalid", + addresses: [{ address: "127.0.0.1", family: 4 }], + privateNetwork: true, + fingerprint: "a".repeat(64), + }; +} + +async function send(port: number, limitOverrides: Partial<LiveRunConfig> = {}) { + const sender = createLabAuthorizedPinnedSender(() => ({})); + return await sender( + {} as LabCredentialLeaseV1, + destination(port), + { address: "127.0.0.1", family: 4 }, + { method: "POST", path: "/", body: "{}" }, + new AbortController().signal, + { ...BASE_LIMITS, ...limitOverrides }, + ); +} + +describe("CL-03 pinned live transport failure classification", () => { + test("preserves first-byte timeout as a transport timeout", async () => { + const port = await listen((_req, res) => { + setTimeout(() => { + if (res.destroyed) return; + res.writeHead(200, { "content-type": "application/json" }); + res.end("{}"); + }, 150); + }); + + await expect(send(port, { firstByteTimeoutMs: 30, inactivityTimeoutMs: 250 })).rejects.toMatchObject({ + name: "TransportError", + code: "first_byte_timeout", + }); + }); + + test("preserves response inactivity as inactivity_timeout", async () => { + const port = await listen((_req, res) => { + res.writeHead(200, { "content-type": "application/json" }); + res.write("{\"ok\":"); + setTimeout(() => { + if (!res.destroyed) res.end("true}"); + }, 150); + }); + + await expect(send(port, { firstByteTimeoutMs: 250, inactivityTimeoutMs: 30 })).rejects.toMatchObject({ + name: "TransportError", + code: "inactivity_timeout", + }); + }); + + test("preserves the output byte ceiling as output_byte_limit", async () => { + const port = await listen((_req, res) => { + res.writeHead(200, { "content-type": "text/plain" }); + res.end("x".repeat(128)); + }); + + await expect(send(port, { maxOutputBytes: 16 })).rejects.toMatchObject({ + name: "TransportError", + code: "output_byte_limit", + }); + }); +}); diff --git a/tests/lab-passive-production-surfaces.test.ts b/tests/lab-passive-production-surfaces.test.ts new file mode 100644 index 0000000000..66ae81fc53 --- /dev/null +++ b/tests/lab-passive-production-surfaces.test.ts @@ -0,0 +1,95 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { mkdirSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { handleLabCommand } from "../src/cli/lab"; +import { + LAB_QUERY_MAX_PAGE_SIZE, + PASSIVE_PRODUCTION_MAX_LIMIT, +} from "../src/lab/query"; +import { handleManagementAPI } from "../src/server/management-api"; +import type { OcxConfig } from "../src/types"; +import { ManagementRequest } from "./helpers/management-auth"; + +const HOMES: string[] = []; +const originalOpenCodexHome = process.env.OPENCODEX_HOME; + +function tempHome(): string { + const dir = join(tmpdir(), `ocx-lab-passive-surfaces-${process.pid}-${Math.random().toString(16).slice(2)}`); + mkdirSync(dir, { recursive: true, mode: 0o700 }); + HOMES.push(dir); + return dir; +} + +afterEach(() => { + for (const dir of HOMES.splice(0)) { + rmSync(dir, { recursive: true, force: true }); + } + if (originalOpenCodexHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = originalOpenCodexHome; +}); + +function config(): OcxConfig { + return { providers: {} } as OcxConfig; +} + +async function apiGet(home: string, path: string): Promise<Response> { + process.env.OPENCODEX_HOME = home; + const req = new ManagementRequest(`http://127.0.0.1${path}`, { method: "GET" }); + const response = await handleManagementAPI(req, new URL(req.url), config(), { + refreshCodexCatalog: async () => {}, + }); + expect(response).not.toBeNull(); + return response!; +} + +describe("CL-09 passive production read surfaces", () => { + test("management API uses the passive query limit without widening generic Lab pages", async () => { + const home = tempHome(); + const subjectId = "a".repeat(64); + + const accepted = await apiGet( + home, + `/api/lab/production-signals?subjectId=${subjectId}&limit=${PASSIVE_PRODUCTION_MAX_LIMIT}`, + ); + expect(accepted.status).toBe(200); + const acceptedBody = await accepted.json() as { + signals: unknown[]; + summary: { recentProductionAttempts: number }; + }; + expect(acceptedBody.signals).toEqual([]); + expect(acceptedBody.summary.recentProductionAttempts).toBe(0); + + const tooHigh = await apiGet( + home, + `/api/lab/production-signals?subjectId=${subjectId}&limit=${PASSIVE_PRODUCTION_MAX_LIMIT + 1}`, + ); + expect(tooHigh.status).toBe(400); + const tooHighBody = await tooHigh.json() as { error: { code: string; message: string } }; + expect(tooHighBody.error.code).toBe("invalid_limit"); + expect(tooHighBody.error.message).toContain(`1 to ${PASSIVE_PRODUCTION_MAX_LIMIT}`); + + const generic = await apiGet(home, `/api/lab/verdicts?limit=${LAB_QUERY_MAX_PAGE_SIZE + 1}`); + expect(generic.status).toBe(400); + const genericBody = await generic.json() as { error: { code: string; message: string } }; + expect(genericBody.error.code).toBe("invalid_limit"); + expect(genericBody.error.message).toContain(`1 to ${LAB_QUERY_MAX_PAGE_SIZE}`); + }); + + test("CLI reports malformed passive subject ids as the actual usage error", async () => { + const home = tempHome(); + const errors: string[] = []; + const originalError = console.error; + console.error = (...args: unknown[]) => { errors.push(args.join(" ")); }; + try { + expect(await handleLabCommand( + ["production-signals", "--subject", "not-a-subject-id"], + { configDir: home }, + )).toBe(2); + expect(errors.join("\n")).toContain("--subject must be an exact Lab route subject id"); + expect(errors.join("\n")).not.toContain("lab read failed"); + } finally { + console.error = originalError; + } + }); +}); diff --git a/tests/lab-private-file-consumer-recovery.test.ts b/tests/lab-private-file-consumer-recovery.test.ts new file mode 100644 index 0000000000..83e6db5a7e --- /dev/null +++ b/tests/lab-private-file-consumer-recovery.test.ts @@ -0,0 +1,70 @@ +import { afterEach, expect, test } from "bun:test"; +import { mkdtempSync, readdirSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { publicEvidenceId } from "../src/lab/public/ids"; +import { isPrivateFileStageName, setPrivateFileCommitFaultForTests } from "../src/lab/public/private-file"; +import { getOrCreatePublicPublisher, signPublicEvidenceBundle } from "../src/lab/public/signature"; +import { storePublicEvidenceBundle } from "../src/lab/public/storage"; +import type { PublicEvidenceRecordV1 } from "../src/lab/public/types"; + +const roots: string[] = []; +afterEach(() => { + setPrivateFileCommitFaultForTests(null); + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function root(): string { + const value = mkdtempSync(join(tmpdir(), "ocx-cl10-recovery-")); + roots.push(value); + return value; +} + +function record(): PublicEvidenceRecordV1 { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const body = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: "2026-08-12", + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", body), ...body }; +} + +test("publisher key recovers after a same-process parent-directory sync failure", () => { + if (process.platform === "win32") return; + const configDir = root(); + setPrivateFileCommitFaultForTests("parent_directory_sync"); + expect(() => getOrCreatePublicPublisher(configDir)).toThrow(/directory.*sync|durab/i); + setPrivateFileCommitFaultForTests(null); + expect(getOrCreatePublicPublisher(configDir).publisher.algorithm).toBe("ed25519"); + expect(readdirSync(join(configDir, "lab")).filter(isPrivateFileStageName)).toEqual([]); +}); + +test("public bundle storage recovers after a same-process parent-directory sync failure", () => { + if (process.platform === "win32") return; + const configDir = root(); + const bundle = signPublicEvidenceBundle({ records: [record()], artifacts: [], createdDayUtc: "2026-08-12", configDir }); + setPrivateFileCommitFaultForTests("parent_directory_sync"); + expect(() => storePublicEvidenceBundle(bundle, configDir)).toThrow(/directory.*sync|durab/i); + setPrivateFileCommitFaultForTests(null); + expect(storePublicEvidenceBundle(bundle, configDir).created).toBe(false); +}); diff --git a/tests/lab-private-file-durability.test.ts b/tests/lab-private-file-durability.test.ts new file mode 100644 index 0000000000..80583aa531 --- /dev/null +++ b/tests/lab-private-file-durability.test.ts @@ -0,0 +1,121 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, utimesSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + cleanupStalePrivateFileStages, + isPrivateFileStageName, + PRIVATE_FILE_STAGE_RETENTION_MS, + publishPrivateFileExclusive, + setPrivateFileCleanupSyncFaultForTests, + setPrivateFileCommitFaultForTests, +} from "../src/lab/public/private-file"; + +const roots: string[] = []; + +afterEach(() => { + setPrivateFileCommitFaultForTests(null); + setPrivateFileCleanupSyncFaultForTests(false); + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function tempRoot(): string { + const root = mkdtempSync(join(tmpdir(), "ocx-cl10-private-file-")); + roots.push(root); + return root; +} + +describe("CL-10 private-file durability", () => { + test("POSIX parent-directory sync failure preserves the published stage until retry", () => { + if (process.platform === "win32") return; + const root = tempRoot(); + const finalPath = join(root, "bundle.json"); + const bytes = Buffer.from("durable-public-evidence", "utf8"); + + setPrivateFileCommitFaultForTests("parent_directory_sync"); + expect(() => publishPrivateFileExclusive(finalPath, bytes)).toThrow(/directory.*sync|durab/i); + expect(existsSync(finalPath)).toBe(true); + expect(readdirSync(root).filter(isPrivateFileStageName)).toHaveLength(1); + + setPrivateFileCommitFaultForTests(null); + expect(publishPrivateFileExclusive(finalPath, bytes)).toEqual({ created: false }); + expect(readFileSync(finalPath).equals(bytes)).toBe(true); + expect(readdirSync(root).filter(isPrivateFileStageName)).toEqual([]); + }); + + test("before-publish failure leaves no final path or staging entry and retry creates cleanly", () => { + const root = tempRoot(); + const finalPath = join(root, "bundle.json"); + const bytes = Buffer.from("durable-public-evidence", "utf8"); + + setPrivateFileCommitFaultForTests("before_publish"); + expect(() => publishPrivateFileExclusive(finalPath, bytes)).toThrow(/before publish/i); + expect(existsSync(finalPath)).toBe(false); + expect(readdirSync(root).filter(isPrivateFileStageName)).toEqual([]); + + setPrivateFileCommitFaultForTests(null); + expect(publishPrivateFileExclusive(finalPath, bytes)).toEqual({ created: true }); + expect(readFileSync(finalPath).equals(bytes)).toBe(true); + expect(readdirSync(root).filter(isPrivateFileStageName)).toEqual([]); + }); + + test("expired published crash witnesses are reclaimed without requiring a retry", () => { + if (process.platform === "win32") return; + const root = tempRoot(); + const finalPath = join(root, "bundle.json"); + const bytes = Buffer.from("durable-public-evidence", "utf8"); + + setPrivateFileCommitFaultForTests("parent_directory_sync"); + expect(() => publishPrivateFileExclusive(finalPath, bytes)).toThrow(/directory.*sync|durab/i); + setPrivateFileCommitFaultForTests(null); + + const stages = readdirSync(root).filter(isPrivateFileStageName); + expect(stages).toHaveLength(1); + const stagePath = join(root, stages[0]!); + const old = new Date(Date.now() - PRIVATE_FILE_STAGE_RETENTION_MS - 60_000); + utimesSync(stagePath, old, old); + + cleanupStalePrivateFileStages(finalPath); + + expect(readFileSync(finalPath).equals(bytes)).toBe(true); + expect(readdirSync(root).filter(isPrivateFileStageName)).toEqual([]); + }); + + test("stale cleanup keeps the crash witness when its pre-unlink directory sync fails", () => { + if (process.platform === "win32") return; + const root = tempRoot(); + const finalPath = join(root, "bundle.json"); + const bytes = Buffer.from("durable-public-evidence", "utf8"); + + setPrivateFileCommitFaultForTests("parent_directory_sync"); + expect(() => publishPrivateFileExclusive(finalPath, bytes)).toThrow(/directory.*sync|durab/i); + setPrivateFileCommitFaultForTests(null); + + const stages = readdirSync(root).filter(isPrivateFileStageName); + expect(stages).toHaveLength(1); + const stagePath = join(root, stages[0]!); + const old = new Date(Date.now() - PRIVATE_FILE_STAGE_RETENTION_MS - 60_000); + utimesSync(stagePath, old, old); + + setPrivateFileCleanupSyncFaultForTests(true); + expect(() => cleanupStalePrivateFileStages(finalPath)).toThrow(/cleanup.*directory.*sync/i); + expect(readFileSync(finalPath).equals(bytes)).toBe(true); + expect(readdirSync(root).filter(isPrivateFileStageName)).toHaveLength(1); + + setPrivateFileCleanupSyncFaultForTests(false); + cleanupStalePrivateFileStages(finalPath); + expect(readdirSync(root).filter(isPrivateFileStageName)).toEqual([]); + }); + + test("Windows publication does not require parent-directory fsync", () => { + if (process.platform !== "win32") return; + const root = tempRoot(); + const finalPath = join(root, "bundle.json"); + const bytes = Buffer.from("durable-public-evidence", "utf8"); + + setPrivateFileCommitFaultForTests("parent_directory_sync"); + expect(publishPrivateFileExclusive(finalPath, bytes)).toEqual({ created: true }); + expect(readFileSync(finalPath).equals(bytes)).toBe(true); + expect(readdirSync(root).filter(isPrivateFileStageName)).toEqual([]); + }); +}); diff --git a/tests/lab-public-api-json.test.ts b/tests/lab-public-api-json.test.ts new file mode 100644 index 0000000000..79d6c2a95e --- /dev/null +++ b/tests/lab-public-api-json.test.ts @@ -0,0 +1,26 @@ +import { describe, expect, test } from "bun:test"; +import { handleManagementAPI } from "../src/server/management-api"; +import type { OcxConfig } from "../src/types"; +import { ManagementRequest } from "./helpers/management-auth"; + +const config = { port: 0, defaultProvider: "openai-apikey", providers: {} } as OcxConfig; + +describe("CL-10 management public JSON boundary", () => { + test("rejects duplicate decoded object keys before request object construction", async () => { + const req = new ManagementRequest("http://127.0.0.1/api/lab/public/community/import", { + method: "POST", + headers: { "content-type": "application/json" }, + body: '{"bundle":{},"\\u0062undle":{}}', + }); + + const response = await handleManagementAPI(req, new URL(req.url), config, { + refreshCodexCatalog: async () => {}, + }); + + expect(response).not.toBeNull(); + expect(response!.status).toBe(400); + expect(await response!.json()).toMatchObject({ + error: { code: "duplicate_json_key" }, + }); + }); +}); diff --git a/tests/lab-public-artifact-policy.test.ts b/tests/lab-public-artifact-policy.test.ts new file mode 100644 index 0000000000..527b96f1c5 --- /dev/null +++ b/tests/lab-public-artifact-policy.test.ts @@ -0,0 +1,33 @@ +import { afterEach, expect, test } from "bun:test"; +import { existsSync, mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { labPublicPublisherKeyPath } from "../src/lab"; +import { publicEvidenceId, signPublicEvidenceBundle } from "../src/lab/public"; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +test("CL-10 local signing rejects artifact bytes without reviewed public_export authority", () => { + const configDir = mkdtempSync(join(tmpdir(), "ocx-cl10-artifact-policy-")); + roots.push(configDir); + const contentBase64 = Buffer.from("credential-canary-1234567890", "utf8").toString("base64"); + const artifact = { + artifactClass: "verifier_summary", + mediaType: "text/plain", + byteCount: Buffer.from(contentBase64, "base64").byteLength, + contentBase64, + }; + const artifactId = publicEvidenceId("artifact", artifact); + + expect(() => signPublicEvidenceBundle({ + records: [], + artifacts: [{ artifactId, ...artifact }], + createdDayUtc: "2026-08-12", + configDir, + })).toThrow(/public_export/i); + + expect(existsSync(labPublicPublisherKeyPath(configDir))).toBe(false); +}); diff --git a/tests/lab-public-coderabbit-regressions.test.ts b/tests/lab-public-coderabbit-regressions.test.ts new file mode 100644 index 0000000000..354203752e --- /dev/null +++ b/tests/lab-public-coderabbit-regressions.test.ts @@ -0,0 +1,230 @@ +import { afterEach, expect, test } from "bun:test"; +import { + existsSync, + mkdirSync, + mkdtempSync, + readdirSync, + rmSync, + unlinkSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { handleLabCommand } from "../src/cli/lab"; +import { purgeSensitiveEvidence } from "../src/lab/ledger/purge"; +import { + ensureLabDirs, + labCommunityDir, + labPublicOriginDir, + labPublicPublisherKeyPath, +} from "../src/lab/paths"; +import { + createPublicEvidenceRevocation, + importCommunityEvidenceBundle, + listCommunityEvidence, + publicEvidenceId, + recordLocalPublicOrigin, + signPublicEvidenceBundle, + writePublicEvidenceBundle, + type PublicEvidenceRecordV1, +} from "../src/lab/public"; +import { + publicEvidenceMutationLockIsReclaimableForTests, + publicEvidenceTryReclaimMutationLockForTests, +} from "../src/lab/public/mutation-lock"; + +const roots: string[] = []; + +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function record(day = "2026-08-12"): PublicEvidenceRecordV1 { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const body = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: day, + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", body), ...body }; +} + +function bundle(home: string, records = [record()]) { + return signPublicEvidenceBundle({ + records, + artifacts: [], + createdDayUtc: "2026-08-14", + configDir: home, + }); +} + +async function captureCli(argv: string[], home: string): Promise<{ code: number; stdout: string; stderr: string }> { + const stdout: string[] = []; + const stderr: string[] = []; + const originalLog = console.log; + const originalError = console.error; + console.log = (...args: unknown[]) => { stdout.push(args.join(" ")); }; + console.error = (...args: unknown[]) => { stderr.push(args.join(" ")); }; + try { + return { + code: await handleLabCommand(argv, { configDir: home }), + stdout: stdout.join("\n"), + stderr: stderr.join("\n"), + }; + } finally { + console.log = originalLog; + console.error = originalError; + } +} + +test("ancient live-PID mutation owner becomes reclaimable after the absolute ceiling", () => { + const home = configDir("ocx-cl10-lock-owner-ceiling-"); + ensureLabDirs(home); + const lockPath = join(labCommunityDir(home), ".mutation-lock"); + mkdirSync(lockPath, { mode: 0o700 }); + const now = Date.now(); + writeFileSync( + join(lockPath, "owner.json"), + JSON.stringify({ + pid: process.pid, + token: "00000000-0000-4000-8000-000000000000", + createdAt: now - (8 * 24 * 60 * 60 * 1000), + }), + { mode: 0o600 }, + ); + + expect(publicEvidenceMutationLockIsReclaimableForTests(home, now)).toBe(true); +}); + +test("ancient live-PID reclaim claim cannot block stale lock recovery forever", () => { + const home = configDir("ocx-cl10-lock-claim-ceiling-"); + ensureLabDirs(home); + const lockPath = join(labCommunityDir(home), ".mutation-lock"); + mkdirSync(lockPath, { mode: 0o700 }); + const now = Date.now(); + const old = now - (8 * 24 * 60 * 60 * 1000); + writeFileSync( + join(lockPath, "owner.json"), + JSON.stringify({ + pid: process.pid, + token: "00000000-0000-4000-8000-000000000000", + createdAt: old, + }), + { mode: 0o600 }, + ); + writeFileSync( + join(lockPath, ".reclaim.json"), + JSON.stringify({ + pid: process.pid, + token: "11111111-1111-4111-8111-111111111111", + createdAt: old, + }), + { mode: 0o600 }, + ); + + expect(publicEvidenceTryReclaimMutationLockForTests(home, now)).toBe(true); + expect(existsSync(lockPath)).toBe(false); +}); + +test("foreign origin-directory entries do not consume marker quota", () => { + const home = configDir("ocx-cl10-origin-foreign-quota-"); + ensureLabDirs(home); + const dir = labPublicOriginDir(home); + for (let index = 0; index < 1024; index += 1) { + writeFileSync(join(dir, `foreign-${String(index).padStart(4, "0")}`), "x", { mode: 0o600 }); + } + const identity = { + publisherKeyId: publicEvidenceId("publisher_key", { seed: "quota-publisher" }), + bundleId: publicEvidenceId("bundle", { seed: "quota-bundle" }), + }; + + expect(() => recordLocalPublicOrigin(identity, home)).not.toThrow(); + expect(existsSync(join(dir, `origin-${identity.publisherKeyId}-${identity.bundleId}.json`))).toBe(true); +}); + +test("corrupt origin classification makes export purge report incomplete instead of clean success", () => { + const home = configDir("ocx-cl10-origin-purge-incomplete-"); + const signed = bundle(home); + const exportPath = writePublicEvidenceBundle(signed, home); + importCommunityEvidenceBundle(signed, home); + const originDir = labPublicOriginDir(home); + const marker = readdirSync(originDir).find((name) => name.startsWith("origin-")); + if (!marker) throw new Error("expected local origin marker"); + writeFileSync(join(originDir, marker), "{", { mode: 0o600 }); + // Remove the two fallback provenance sources so the corrupt marker is the only + // evidence that can classify the matching community copy as locally originated. + writeFileSync(exportPath, "{", { mode: 0o600 }); + unlinkSync(labPublicPublisherKeyPath(home)); + + expect(() => purgeSensitiveEvidence({ + configDir: home, + purgeActions: ["export"], + recordedAt: Date.UTC(2026, 7, 14, 19, 0, 0), + })).toThrow(/origin|classification|incomplete/i); + expect(existsSync(exportPath)).toBe(false); + expect(listCommunityEvidence(home).map((row) => row.bundleId)).toContain(signed.bundleId); +}); + +test("failed CLI verification is a state failure and never prints command usage", async () => { + const home = configDir("ocx-cl10-cli-verify-failure-"); + const signed = bundle(home); + const tampered = { ...signed, bundleDigest: "0".repeat(64) }; + const path = join(home, "tampered.json"); + writeFileSync(path, JSON.stringify(tampered), { mode: 0o600 }); + + const result = await captureCli(["public", "verify", "--file", path], home); + expect(result.code).toBe(1); + expect(result.stdout).toMatch(/digest_invalid/i); + expect(result.stderr).toMatch(/verification failed.*digest_invalid/i); + expect(result.stderr).not.toMatch(/Usage:/i); +}); + +test("revocation canonicalization does not depend on localeCompare", () => { + const home = configDir("ocx-cl10-revocation-order-"); + const first = record("2026-08-12"); + const second = record("2026-08-13"); + const signed = bundle(home, [first, second]); + const originalLocaleCompare = String.prototype.localeCompare; + String.prototype.localeCompare = function localeCompareForbidden(): number { + throw new Error("localeCompare must not participate in signed canonicalization"); + }; + try { + expect(() => createPublicEvidenceRevocation({ + configDir: home, + targetBundle: signed, + issuedDayUtc: "2026-08-14", + reason: "superseded", + targets: [ + { kind: "record", id: second.recordId }, + { kind: "record", id: first.recordId }, + ], + })).not.toThrow(); + } finally { + String.prototype.localeCompare = originalLocaleCompare; + } +}); diff --git a/tests/lab-public-core-contract.test.ts b/tests/lab-public-core-contract.test.ts new file mode 100644 index 0000000000..9705d5e372 --- /dev/null +++ b/tests/lab-public-core-contract.test.ts @@ -0,0 +1,221 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { chmodSync, existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { labPublicPublisherKeyPath } from "../src/lab/paths"; +import { buildPublicEvidenceBundle } from "../src/lab/public/bundle"; +import { validatePublicEvidenceAuthorities } from "../src/lab/public/community-authority"; +import { publicEvidenceId } from "../src/lab/public/ids"; +import { validatePublicEvidencePrivacy } from "../src/lab/public/privacy"; +import { PUBLIC_ROUTE_REGISTRY_V1 } from "../src/lab/public/registry"; +import { signPublicEvidenceBundle, verifyPublicEvidenceBundle } from "../src/lab/public/signature"; +import { parseStrictPublicJson } from "../src/lab/public/strict-json"; +import { publicUtcDay } from "../src/lab/public/time"; +import { + PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION, + type PublicEvidenceBundleUnsignedV1, +} from "../src/lab/public/types"; +import { validatePublicRouteRegistryManifest } from "../src/lab/public/validate"; + +const FIXED_PRIVATE_KEY = [ + `-----BEGIN PRIVATE ${"KEY"}-----`, + ["MC4CAQAwBQYDK2VwBCIEIAABAgMEBQYH", "CAkKCwwNDg8QERITFBUWFxgZGhscHR4f"].join(""), + `-----END PRIVATE ${"KEY"}-----`, + "", +].join("\n"); +const FIXED_PUBLIC_KEY = "MCowBQYDK2VwAyEAA6EHv/POEL4dcN0Y50vAmWfk1jCbpQ1fHdyGZBJVMbg="; +const REVIEWED_AUTHORITY_SOURCE_COMMIT = "75a21417657ba5a3033198be0d8ae949de723d11"; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function installFixedPublisherKey(config: string): void { + const path = labPublicPublisherKeyPath(config); + mkdirSync(join(path, ".."), { recursive: true, mode: 0o700 }); + writeFileSync(path, FIXED_PRIVATE_KEY, { encoding: "utf8", mode: 0o600 }); + if (process.platform !== "win32") chmodSync(path, 0o600); +} + +function fixedRecord(scenarioId = "responses-core.protocol.request-shape") { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const withoutRecordId = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId, + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: "2026-08-12", + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId }; +} + +describe("CL-10 public evidence core contract", () => { + test("freezes the RFC 8785/domain-separated bundle and Ed25519 vector", () => { + const config = configDir("ocx-cl10-core-wire-"); + installFixedPublisherKey(config); + const bundle = signPublicEvidenceBundle({ + records: [fixedRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + configDir: config, + }); + + expect(bundle.publisher.publicKey).toBe(FIXED_PUBLIC_KEY); + expect(bundle.publisher.keyId).toBe("4d5a347afcc7a1ac8d2dd4e573f0fbca2d2e90dd472c35df5c72bf2d2afca08f"); + expect(bundle.records[0]!.recordId).toBe("5bec20821bbf01f831e74ba469e7f18481c1209fdef209c76f482105de3e406d"); + expect(bundle.bundleId).toBe("a7598b68a4cf884dc381b1d88111e74bfad5e74ceae2be8de55b88bac3250401"); + expect(bundle.bundleDigest).toBe("aeef2f3e64a131588f6a34aaea1172c352a0c803f838690c2d6ee652ca74fb87"); + expect(bundle.signature.signature).toBe("UAiI7Mz4/yIU5XjSuNZFSuyFPoAvGCy+x9cpTCwYKnFDq20AP6ipV3zowD3S4KP2iYfkXyHTMsMH3CEnz6lCBw=="); + expect(verifyPublicEvidenceBundle(bundle)).toEqual({ status: "cryptographically_valid" }); + }); + + test("pins canonical multi-record ordering into the bundle digest", () => { + const first = fixedRecord(); + const second = fixedRecord("responses-core.protocol.response-shape"); + const publisher = { + algorithm: "ed25519" as const, + publicKey: FIXED_PUBLIC_KEY, + keyId: publicEvidenceId("publisher_key", { algorithm: "ed25519", publicKey: FIXED_PUBLIC_KEY }), + }; + + const bundle = buildPublicEvidenceBundle({ + records: [first, second], + artifacts: [], + createdDayUtc: "2026-08-12", + publisher, + }); + + expect(bundle.records.map((record) => record.recordId)).toEqual([ + "2a2a2e8406e6ccac915b21e96558a7b89e49e52effe474bd2c861ad2f7459437", + "5bec20821bbf01f831e74ba469e7f18481c1209fdef209c76f482105de3e406d", + ]); + expect(bundle.bundleDigest).toBe("63fef67418ec196b480bba3865fba287cc92aa94a760e2e3648b0759c0be046e"); + }); + + test("retains protocol V1 authority as an explicit historical snapshot", () => { + const current = fixedRecord(); + expect(() => validatePublicEvidenceAuthorities([current])).not.toThrow(); + + const { recordId: _recordId, ...body } = current; + const unsupportedBody = { + ...body, + suiteVersion: "9.9.9", + scenarioVersion: "9.9.9", + }; + const unsupported = { + recordId: publicEvidenceId("record", unsupportedBody), + ...unsupportedBody, + }; + expect(() => validatePublicEvidenceAuthorities([unsupported])).toThrow(/not retained/i); + }); + + test("rejects non-canonical publisher Base64", () => { + const publicKey = `${FIXED_PUBLIC_KEY}\n`; + const publisher = { + algorithm: "ed25519" as const, + publicKey, + keyId: publicEvidenceId("publisher_key", { algorithm: "ed25519", publicKey }), + }; + expect(() => buildPublicEvidenceBundle({ + records: [fixedRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + publisher, + })).toThrow(/canonical base64/i); + }); + + test("strict JSON rejects duplicate decoded keys and bound violations before materialization", () => { + expect(() => parseStrictPublicJson(Buffer.from('{"a":1,"\\u0061":2}', "utf8"))) + .toThrow(/duplicate json object key/i); + expect(() => parseStrictPublicJson(Buffer.from(`${"[".repeat(9)}0${"]".repeat(9)}`, "utf8"))) + .toThrow(/nesting depth exceeds 8/i); + expect(() => parseStrictPublicJson(Buffer.from(`[${Array.from({ length: 513 }, () => "0").join(",")}]`, "utf8"))) + .toThrow(/array exceeds 512/i); + const wide = `{${Array.from({ length: 65 }, (_, index) => `"k${index}":0`).join(",")}}`; + expect(() => parseStrictPublicJson(Buffer.from(wide, "utf8"))).toThrow(/object exceeds 64/i); + expect(() => parseStrictPublicJson(Buffer.alloc((2 * 1024 * 1024) + 1, 0x20))) + .toThrow(/exceeds 2097152 bytes/i); + }); + + test("public UTC day rejects expanded-year timestamps", () => { + expect(() => publicUtcDay(Date.UTC(10_000, 0, 1))).toThrow(/completion timestamp/i); + }); + + test("local signing rejects artifact bytes before creating publisher state", () => { + const config = configDir("ocx-cl10-core-artifact-"); + const contentBase64 = Buffer.from("credential-canary-1234567890", "utf8").toString("base64"); + const artifact = { + artifactClass: "verifier_summary", + mediaType: "text/plain", + byteCount: Buffer.from(contentBase64, "base64").byteLength, + contentBase64, + }; + const artifactId = publicEvidenceId("artifact", artifact); + expect(() => signPublicEvidenceBundle({ + records: [], + artifacts: [{ artifactId, ...artifact }], + createdDayUtc: "2026-08-12", + configDir: config, + })).toThrow(/public_export/i); + expect(existsSync(labPublicPublisherKeyPath(config))).toBe(false); + }); + + test("privacy rejects embedded unbracketed IPv6 in artifact text", () => { + const bytes = Buffer.from("artifact 2001:db8::1 content", "utf8"); + const bundle = { + createdDayUtc: "2026-08-13", + records: [], + artifacts: [{ + artifactId: "0".repeat(64), + artifactClass: "verifier_summary", + mediaType: "text/plain", + byteCount: bytes.byteLength, + contentBase64: bytes.toString("base64"), + }], + } as unknown as PublicEvidenceBundleUnsignedV1; + expect(() => validatePublicEvidencePrivacy(bundle)).toThrow(/IP address|privacy/i); + }); + + test("pins the reviewed public route registry authority", () => { + const manifest = validatePublicRouteRegistryManifest(PUBLIC_ROUTE_REGISTRY_V1); + expect(manifest.schemaVersion).toBe(PUBLIC_ROUTE_REGISTRY_SCHEMA_VERSION); + expect(manifest.registryVersion).toBe("2026-08-13.v2"); + expect(manifest.sourceCommit).toBe(REVIEWED_AUTHORITY_SOURCE_COMMIT); + expect(manifest.entries).toEqual([{ + providerId: "openai", + modelId: "gpt-5.6-sol", + adapterFamilies: ["openai-responses"], + }]); + + expect(Object.isFrozen(PUBLIC_ROUTE_REGISTRY_V1)).toBe(true); + expect(Object.isFrozen(PUBLIC_ROUTE_REGISTRY_V1.entries)).toBe(true); + for (const entry of PUBLIC_ROUTE_REGISTRY_V1.entries) { + expect(Object.isFrozen(entry)).toBe(true); + expect(Object.isFrozen(entry.adapterFamilies)).toBe(true); + } + }); +}); diff --git a/tests/lab-public-deep-review-regressions.test.ts b/tests/lab-public-deep-review-regressions.test.ts new file mode 100644 index 0000000000..d2cd112d62 --- /dev/null +++ b/tests/lab-public-deep-review-regressions.test.ts @@ -0,0 +1,250 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { jcsStringify } from "../src/lab/conformance/jcs"; +import { labCommunityDir, labPublicPublisherKeyPath } from "../src/lab/paths"; +import { + buildPublicEvidenceBundle, + createPublicEvidenceRevocation, + getOrCreatePublicPublisher, + importCommunityEvidenceBundle, + importCommunityEvidenceRevocation, + listCommunityEvidence, + parseStrictPublicJson, + publicEvidenceId, + signPublicEvidenceBundle, + signPublicPublisherDigest, + validatePublicEvidenceRecordPrivacy, + verifyPublicEvidenceBundle, + type PublicArtifactV1, + type PublicEvidenceBundleV1, + type PublicEvidenceRecordV1, +} from "../src/lab/public"; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function fixedRecord(overrides: Partial<Omit<PublicEvidenceRecordV1, "recordId" | "subjectId" | "subject">> = {}): PublicEvidenceRecordV1 { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const withoutRecordId = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: "2026-08-12", + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + ...overrides, + }; + return { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId }; +} + +function rebuildRecord(record: PublicEvidenceRecordV1, patch: Partial<Omit<PublicEvidenceRecordV1, "recordId">>): PublicEvidenceRecordV1 { + const { recordId: _recordId, ...base } = record; + const withoutRecordId = { ...base, ...patch }; + return { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId } as PublicEvidenceRecordV1; +} + +function signArbitraryBundle(input: { + configDir: string; + records: PublicEvidenceRecordV1[]; + artifacts?: PublicArtifactV1[]; + createdDayUtc?: string; +}): PublicEvidenceBundleV1 { + const handle = getOrCreatePublicPublisher(input.configDir); + const unsigned = buildPublicEvidenceBundle({ + records: input.records, + artifacts: input.artifacts ?? [], + createdDayUtc: input.createdDayUtc ?? "2026-08-12", + publisher: handle.publisher, + }); + return { + ...unsigned, + signature: { + algorithm: "ed25519", + signedDigest: unsigned.bundleDigest, + signature: signPublicPublisherDigest(handle, unsigned.bundleDigest), + }, + }; +} + +function publicArtifact(content: string): PublicArtifactV1 { + const contentBase64 = Buffer.from(content, "utf8").toString("base64"); + const body = { + artifactClass: "verifier_summary", + mediaType: "text/plain", + byteCount: Buffer.from(contentBase64, "base64").byteLength, + contentBase64, + }; + return { artifactId: publicEvidenceId("artifact", body), ...body }; +} + +describe("CL-10 deep-review trust regressions", () => { + test("verification rejects a signed bundle whose canonical record order was changed", () => { + const publisher = configDir("ocx-cl10-order-"); + const first = fixedRecord(); + const second = fixedRecord({ observedDayUtc: "2026-08-13" }); + const bundle = signArbitraryBundle({ configDir: publisher, records: [first, second] }); + expect(bundle.records).toHaveLength(2); + + const reordered = { ...bundle, records: [...bundle.records].reverse() }; + expect(reordered.records.map(row => row.recordId)).not.toEqual(bundle.records.map(row => row.recordId)); + expect(verifyPublicEvidenceBundle(reordered)).toEqual({ status: "schema_rejected" }); + }); + + test("community import rejects artifact bytes until reviewed public_export authority exists", () => { + const publisher = configDir("ocx-cl10-artifact-publisher-"); + const consumer = configDir("ocx-cl10-artifact-consumer-"); + const bundle = signArbitraryBundle({ + configDir: publisher, + records: [fixedRecord()], + artifacts: [publicArtifact("synthetic-safe-content")], + }); + expect(verifyPublicEvidenceBundle(bundle)).toEqual({ status: "cryptographically_valid" }); + expect(() => importCommunityEvidenceBundle(bundle, consumer)).toThrow(/public_export|artifact.*authority/i); + }); + + test("record revocation remains effective when the same publisher later imports another bundle containing that record", () => { + const publisher = configDir("ocx-cl10-revoke-publisher-"); + const consumer = configDir("ocx-cl10-revoke-consumer-"); + const record = fixedRecord(); + const first = signPublicEvidenceBundle({ records: [record], artifacts: [], createdDayUtc: "2026-08-12", configDir: publisher }); + const second = signPublicEvidenceBundle({ records: [record], artifacts: [], createdDayUtc: "2026-08-13", configDir: publisher }); + expect(first.bundleId).not.toBe(second.bundleId); + + importCommunityEvidenceBundle(first, consumer); + const revocation = createPublicEvidenceRevocation({ + configDir: publisher, + targetBundle: first, + issuedDayUtc: "2026-08-13", + reason: "evidence_invalidated", + targets: [{ kind: "record", id: record.recordId }], + }); + importCommunityEvidenceRevocation(revocation, consumer); + importCommunityEvidenceBundle(second, consumer); + + const summaries = listCommunityEvidence(consumer); + expect(summaries.map((row) => row.bundleId)).toEqual([first.bundleId, second.bundleId].sort()); + expect(summaries).toEqual(expect.arrayContaining([ + expect.objectContaining({ bundleId: first.bundleId, activeRecordCount: 0, revokedRecordCount: 1 }), + expect.objectContaining({ bundleId: second.bundleId, activeRecordCount: 0, revokedRecordCount: 1 }), + ])); + }); + + test("invalid signing input fails before publisher identity is created", () => { + const home = configDir("ocx-cl10-invalid-sign-"); + expect(() => signPublicEvidenceBundle({ + records: [fixedRecord()], + artifacts: [], + createdDayUtc: "not-a-day", + configDir: home, + })).toThrow(/day/i); + expect(existsSync(labPublicPublisherKeyPath(home))).toBe(false); + }); + + test("foreign revocation attempt does not create a new publisher identity", () => { + const publisher = configDir("ocx-cl10-foreign-target-"); + const attacker = configDir("ocx-cl10-foreign-revoker-"); + const target = signPublicEvidenceBundle({ + records: [fixedRecord()], artifacts: [], createdDayUtc: "2026-08-12", configDir: publisher, + }); + expect(() => createPublicEvidenceRevocation({ + configDir: attacker, + targetBundle: target, + issuedDayUtc: "2026-08-13", + reason: "publisher_retracted", + targets: [{ kind: "bundle", id: target.bundleId }], + })).toThrow(/publisher|key/i); + expect(existsSync(labPublicPublisherKeyPath(attacker))).toBe(false); + }); + + test("JCS rejects lone UTF-16 surrogate code units", () => { + expect(() => jcsStringify("\uDEAD")).toThrow(/unicode|surrogate/i); + expect(() => jcsStringify({ ["\uDEAD"]: true })).toThrow(/unicode|surrogate/i); + }); + + test("reviewed assertion authority requires exact unique assertion coverage", () => { + const missingHome = configDir("ocx-cl10-assert-missing-"); + const duplicateHome = configDir("ocx-cl10-assert-duplicate-"); + const base = fixedRecord(); + const missing = rebuildRecord(base, { assertions: [] }); + const duplicate = rebuildRecord(base, { assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + { id: "method", required: true, passed: false }, + ] }); + + expect(() => signPublicEvidenceBundle({ + records: [missing], artifacts: [], createdDayUtc: "2026-08-12", configDir: missingHome, + })).toThrow(/assertion.*authority|missing.*assertion/i); + expect(existsSync(labPublicPublisherKeyPath(missingHome))).toBe(false); + + expect(() => signPublicEvidenceBundle({ + records: [duplicate], artifacts: [], createdDayUtc: "2026-08-12", configDir: duplicateHome, + })).toThrow(/assertion.*authority|duplicate.*assertion/i); + expect(existsSync(labPublicPublisherKeyPath(duplicateHome))).toBe(false); + }); + + test("community import enforces the cache file quota before creating another object", () => { + const publisher = configDir("ocx-cl10-cache-publisher-"); + const consumer = configDir("ocx-cl10-cache-consumer-"); + const community = labCommunityDir(consumer); + mkdirSync(community, { recursive: true, mode: 0o700 }); + for (let index = 0; index < 512; index += 1) { + writeFileSync(join(community, `occupied-${String(index).padStart(3, "0")}`), "x", { mode: 0o600 }); + } + const bundle = signPublicEvidenceBundle({ + records: [fixedRecord()], artifacts: [], createdDayUtc: "2026-08-12", configDir: publisher, + }); + expect(() => importCommunityEvidenceBundle(bundle, consumer)).toThrow(/cache.*bound|cache.*limit|capacity/i); + }); + + test("duplicate-key diagnostics are bounded and do not reflect attacker-controlled key contents", () => { + const key = `SECRET-${"x".repeat(64 * 1024)}`; + const raw = Buffer.from(`{${JSON.stringify(key)}:1,${JSON.stringify(key)}:2}`, "utf8"); + let failure: unknown; + try { + parseStrictPublicJson(raw); + } catch (error) { + failure = error; + } + expect(failure).toBeInstanceOf(Error); + const message = (failure as Error).message; + expect(message).toMatch(/duplicate json object key/i); + expect(message.length).toBeLessThan(256); + expect(message).not.toContain("SECRET-"); + }); + + test("privacy scanner rejects unbracketed IPv6 literals", () => { + const base = fixedRecord(); + const subject = { ...base.subject, surface: "2001:db8::1" }; + const subjectId = publicEvidenceId("subject", subject); + const record = rebuildRecord(base, { subject, subjectId }); + expect(() => validatePublicEvidenceRecordPrivacy(record)).toThrow(/IP address|privacy/i); + }); +}); diff --git a/tests/lab-public-evidence.test.ts b/tests/lab-public-evidence.test.ts new file mode 100644 index 0000000000..ec8302183e --- /dev/null +++ b/tests/lab-public-evidence.test.ts @@ -0,0 +1,355 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { existsSync, mkdirSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + LAB_EVENT_SCHEMA_VERSION, + LAB_PRODUCER, + assignEventId, + subjectIdForSubject, + type ObservationEvent, + type ProtocolSubjectV1, + type RouteSubjectV1, +} from "../src/lab"; +import { labPublicPublisherKeyPath } from "../src/lab/paths"; +import { + PUBLIC_ROUTE_REGISTRY_V1, + PublicEvidenceValidationError, + buildPublicEvidenceBundle, + getOrCreatePublicPublisher, + isPublicIncidentRef, + projectPublicEvidence, + projectPublicEvidenceRecord, + publicEvidenceId, + readPublicEvidenceBundle, + signPublicEvidenceBundle, + validatePublicEvidenceRecord, + validatePublicRouteRegistryManifest, + verifyPublicEvidenceBundle, + writePublicEvidenceBundle, +} from "../src/lab/public"; + +const HOMES: string[] = []; +const DEFAULT_COMPLETED_AT = Date.UTC(2026, 7, 12, 14, 37, 41); + +function tempHome(): string { + const dir = join(tmpdir(), `ocx-lab-public-${process.pid}-${Math.random().toString(16).slice(2)}`); + mkdirSync(dir, { recursive: true, mode: 0o700 }); + HOMES.push(dir); + return dir; +} + +afterEach(() => { + for (const dir of HOMES.splice(0)) { + try { + rmSync(dir, { recursive: true, force: true }); + } catch { + /* ignore */ + } + } +}); + +function hex(seed: string): string { + return Bun.CryptoHasher.hash("sha256", seed, "hex"); +} + +function protocolObservation(completedAt = DEFAULT_COMPLETED_AT): ObservationEvent { + const subject: ProtocolSubjectV1 = { + subjectSchemaVersion: 1, + subjectKind: "protocol", + opencodexCompatibilityVersion: "2.13.0", + effectiveAdapter: "openai-chat", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + behaviorFingerprint: hex("private-protocol-behavior"), + }; + const subjectId = subjectIdForSubject(subject); + return assignEventId({ + schemaVersion: LAB_EVENT_SCHEMA_VERSION, + eventKind: "observation" as const, + recordedAt: completedAt + 7_000, + producer: LAB_PRODUCER, + producerVersion: "2.13.0", + evidenceLayer: "protocol_conformance" as const, + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + scenarioManifestDigest: hex("scenario"), + suiteId: "responses-core", + suiteVersion: "1.0.0", + suiteManifestDigest: hex("suite"), + fixtureDigests: [hex("fixture")], + subject, + subjectId, + startedAt: completedAt - 1_000, + completedAt, + executionMode: "fixture" as const, + attempt: 1, + limits: { totalTimeoutMs: 1000 }, + outcome: "pass" as const, + assertions: [ + { + id: "method", + operator: "equals", + required: true, + passed: true, + expectedSummary: "CANARY-PRIVATE-EXPECTED", + observedSummary: "CANARY-PRIVATE-OBSERVED", + }, + { id: "message", operator: "equals", required: true, passed: true }, + { id: "temperature", operator: "equals", required: true, passed: true }, + ], + environment: { localPath: "C:\\Users\\private\\repo" }, + artifactRefs: [], + sourceRefs: ["request_1234567890", "decision_1234567890"], + }) as ObservationEvent; +} + +function routeObservation(completedAt = DEFAULT_COMPLETED_AT): ObservationEvent { + const subject: RouteSubjectV1 = { + subjectSchemaVersion: 1, + subjectKind: "route", + providerId: "openai", + providerInstanceFingerprint: hex("PRIVATE-provider-instance"), + clientModelId: "gpt-5.6-sol", + upstreamModelId: "gpt-5.6-sol", + effectiveAdapter: "openai-responses", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-responses", + surface: "responses-http", + opencodexCompatibilityVersion: "2.13.0", + behaviorFingerprint: hex("PRIVATE-route-behavior"), + endpointFingerprint: hex("PRIVATE-endpoint"), + dependencies: [], + }; + const subjectId = subjectIdForSubject(subject); + return assignEventId({ + ...protocolObservation(completedAt), + eventId: undefined, + evidenceLayer: "live_route_compatibility" as const, + scenarioId: "responses-core.live.request-shape", + executionMode: "live" as const, + subject, + subjectId, + sourceRefs: ["request_PRIVATE", "decision_PRIVATE"], + }) as ObservationEvent; +} + +function exportedProtocolRecord() { + const result = projectPublicEvidenceRecord({ observation: protocolObservation(), verdict: "VERIFIED" }); + if (result.status !== "exportable") throw new Error("expected exportable protocol record"); + return result.record; +} + +function withRecomputedRecordId(record: ReturnType<typeof exportedProtocolRecord>) { + const { recordId: _oldRecordId, ...withoutRecordId } = record; + return { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId }; +} + +describe("CL-10 public authority", () => { + test("ships a closed, self-consistent public route registry manifest", () => { + const manifest = validatePublicRouteRegistryManifest(PUBLIC_ROUTE_REGISTRY_V1); + expect(manifest.schemaVersion).toBe("public_route_registry_v1"); + expect(manifest.entries.length).toBeGreaterThan(0); + expect(manifest.manifestDigest).toMatch(/^[0-9a-f]{64}$/); + expect(manifest.entries.every((entry) => entry.providerId && entry.modelId)).toBe(true); + }); + + test("public incident references are closed corpus ids only", () => { + expect(isPublicIncidentRef("IC-001")).toBe(true); + expect(isPublicIncidentRef("IC-020")).toBe(true); + expect(isPublicIncidentRef("https://github.com/private/issue/1")).toBe(false); + expect(isPublicIncidentRef("devlog/_plan/private.md")).toBe(false); + expect(isPublicIncidentRef("IC-1")).toBe(false); + }); +}); + +describe("CL-10 public projection", () => { + test("projects protocol evidence without leaking local ids, diagnostics, or assertion text", () => { + const event = protocolObservation(); + const result = projectPublicEvidenceRecord({ observation: event, verdict: "VERIFIED" }); + expect(result.status).toBe("exportable"); + if (result.status !== "exportable") throw new Error("expected exportable protocol record"); + + expect(result.record.evidenceLayer).toBe("protocol_conformance"); + expect(result.record.subject.subjectKind).toBe("protocol"); + expect(result.record.observedDayUtc).toBe("2026-08-12"); + expect(result.record.subjectId).toMatch(/^[0-9a-f]{64}$/); + expect(result.record.subjectId).not.toBe(event.subjectId); + expect(result.record.recordId).toMatch(/^[0-9a-f]{64}$/); + expect(result.record.assertions).toEqual([ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ]); + + const serialized = JSON.stringify(result.record); + for (const canary of [ + event.subjectId, + event.eventId, + "CANARY-PRIVATE-EXPECTED", + "CANARY-PRIVATE-OBSERVED", + "C:\\Users\\private\\repo", + "request_1234567890", + "decision_1234567890", + (event.subject as ProtocolSubjectV1).behaviorFingerprint, + ]) { + expect(serialized).not.toContain(canary); + } + }); + + test("does not generalise a private exact route into a public claim", () => { + const event = routeObservation(); + const result = projectPublicEvidenceRecord({ observation: event, verdict: "PROBED" }); + expect(result).toEqual({ status: "not_exportable", reason: "private_route_identity" }); + }); + + test("derives bundle day only from records that survive exportability gates", () => { + const olderPublic = protocolObservation(Date.UTC(2026, 7, 12, 23, 59, 59)); + const newerPrivateRoute = routeObservation(Date.UTC(2026, 7, 13, 12, 0, 0)); + const projected = projectPublicEvidence({ + createdDayUtc: "2099-12-31", + records: [ + { observation: olderPublic, verdict: "VERIFIED" }, + { observation: newerPrivateRoute, verdict: "PROBED" }, + ], + }); + expect(projected.bundle.createdDayUtc).toBe("2026-08-12"); + expect(projected.bundle.records).toHaveLength(1); + expect(projected.excluded).toEqual([{ index: 1, reason: "private_route_identity" }]); + }); + + test("uses domain-separated deterministic public ids", () => { + const payload = { providerId: "openai", modelId: "gpt-5.6-sol" }; + const a = publicEvidenceId("subject", payload); + const b = publicEvidenceId("subject", payload); + const c = publicEvidenceId("record", payload); + expect(a).toBe(b); + expect(a).toMatch(/^[0-9a-f]{64}$/); + expect(a).not.toBe(c); + }); + + test("runtime validation rejects unknown public fields", () => { + const result = projectPublicEvidenceRecord({ observation: protocolObservation(), verdict: "VERIFIED" }); + if (result.status !== "exportable") throw new Error("expected exportable protocol record"); + const withUnknown = { ...result.record, localSubjectId: "PRIVATE" }; + expect(() => validatePublicEvidenceRecord(withUnknown)).toThrow(PublicEvidenceValidationError); + }); +}); + +describe("CL-10 public bundle and publisher", () => { + test("builds deterministic bundle ids and digests from public-safe bytes", () => { + const home = tempHome(); + const publisher = getOrCreatePublicPublisher(home).publisher; + const input = { + records: [exportedProtocolRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + publisher, + }; + const a = buildPublicEvidenceBundle(input); + const b = buildPublicEvidenceBundle(input); + expect(a.bundleId).toBe(b.bundleId); + expect(a.bundleDigest).toBe(b.bundleDigest); + expect(a.bundleId).toMatch(/^[0-9a-f]{64}$/); + expect(a.bundleDigest).toMatch(/^[0-9a-f]{64}$/); + expect(a.bundleId).not.toBe(a.bundleDigest); + }); + + test("creates one installation-local Ed25519 publisher key with restrictive permissions", () => { + const home = tempHome(); + const first = getOrCreatePublicPublisher(home); + const second = getOrCreatePublicPublisher(home); + expect(first.publisher).toEqual(second.publisher); + expect(first.publisher.algorithm).toBe("ed25519"); + expect(first.publisher.keyId).toMatch(/^[0-9a-f]{64}$/); + expect(first.publisher.publicKey.length).toBeGreaterThan(20); + const privateKey = readFileSync(first.privateKeyPath, "utf8"); + expect(privateKey).toContain("PRIVATE KEY"); + if (process.platform !== "win32") { + expect(statSync(first.privateKeyPath).mode & 0o777).toBe(0o600); + } + }); + + test("rejects unreviewed assertion authority before publisher key creation", () => { + const home = tempHome(); + const record = exportedProtocolRecord(); + const unauthorized = withRecomputedRecordId({ + ...record, + assertions: [{ id: "private-assertion-name", required: true, passed: true }], + }); + expect(() => signPublicEvidenceBundle({ + records: [unauthorized], + artifacts: [], + createdDayUtc: "2026-08-12", + configDir: home, + })).toThrow(/assertion.*authority/i); + expect(existsSync(labPublicPublisherKeyPath(home))).toBe(false); + }); + + test("rejects privacy-canary public fields before publisher key creation", () => { + const home = tempHome(); + const record = exportedProtocolRecord(); + if (record.subject.subjectKind !== "protocol") throw new Error("expected protocol public subject"); + const subject = { ...record.subject, surface: "https://private.example.test/path?token=secret" }; + const subjectId = publicEvidenceId("subject", subject); + const unsafe = withRecomputedRecordId({ ...record, subject, subjectId }); + expect(() => signPublicEvidenceBundle({ + records: [unsafe], + artifacts: [], + createdDayUtc: "2026-08-12", + configDir: home, + })).toThrow(/closed public identifier|forbidden URL material/i); + expect(existsSync(labPublicPublisherKeyPath(home))).toBe(false); + }); + + test("signs and verifies exact canonical bundle bytes without serializing private key material", () => { + const home = tempHome(); + const handle = getOrCreatePublicPublisher(home); + const bundle = signPublicEvidenceBundle({ + records: [exportedProtocolRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + configDir: home, + }); + expect(verifyPublicEvidenceBundle(bundle)).toEqual({ status: "cryptographically_valid" }); + const serialized = JSON.stringify(bundle); + expect(serialized).not.toContain(handle.privateKeyPath); + const pemBodyLines = readFileSync(handle.privateKeyPath, "utf8") + .split("\n") + .map((line) => line.trim()) + .filter((line) => line.length > 0 && !line.startsWith("-----")); + expect(pemBodyLines.length).toBeGreaterThan(0); + for (const line of pemBodyLines) expect(serialized).not.toContain(line); + + const badDigest = { ...bundle, bundleDigest: hex("tampered-bundle") }; + expect(verifyPublicEvidenceBundle(badDigest)).toEqual({ status: "digest_invalid" }); + const badSignature = { + ...bundle, + signature: { ...bundle.signature, signature: Buffer.from("tampered").toString("base64") }, + }; + expect(verifyPublicEvidenceBundle(badSignature)).toEqual({ status: "signature_invalid" }); + }); + + test("writes and reads a bounded local export by public bundle id", () => { + const home = tempHome(); + const bundle = signPublicEvidenceBundle({ + records: [exportedProtocolRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + configDir: home, + }); + const path = writePublicEvidenceBundle(bundle, home); + expect(path).toBe(join(home, "lab", "export", `${bundle.bundleId}.json`)); + expect(readPublicEvidenceBundle(bundle.bundleId, home)).toEqual(bundle); + }); + + test("rejects non-object local export JSON with a validation error", () => { + const home = tempHome(); + const bundleId = "f".repeat(64); + const exportDir = join(home, "lab", "export"); + mkdirSync(exportDir, { recursive: true, mode: 0o700 }); + writeFileSync(join(exportDir, `${bundleId}.json`), "null\n", { encoding: "utf8", mode: 0o600 }); + expect(() => readPublicEvidenceBundle(bundleId, home)).toThrow(PublicEvidenceValidationError); + }); +}); \ No newline at end of file diff --git a/tests/lab-public-export-transaction.test.ts b/tests/lab-public-export-transaction.test.ts new file mode 100644 index 0000000000..4a99e341b8 --- /dev/null +++ b/tests/lab-public-export-transaction.test.ts @@ -0,0 +1,103 @@ +import { afterEach, expect, test } from "bun:test"; +import { existsSync, mkdirSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + labExportDir, + labPublicOriginDir, + persistConformanceResult, + rebuildLabProjection, +} from "../src/lab"; +import { createArtifactStore } from "../src/lab/artifacts/store"; +import { resolveProtocolExecutionContext } from "../src/lab/conformance/executor"; +import { discoverScenarios, loadCaseAuthority } from "../src/lab/conformance/manifest"; +import type { CaseRecord } from "../src/lab/conformance/types"; +import { + buildPublicEvidenceBundle, + exportLocalPublicEvidence, + getOrCreatePublicPublisher, + previewLocalPublicEvidence, +} from "../src/lab/public"; +import { queryLabObservations } from "../src/lab/query"; + +const homes: string[] = []; + +afterEach(() => { + for (const home of homes.splice(0)) rmSync(home, { recursive: true, force: true }); +}); + +function tempHome(): string { + const home = join(tmpdir(), `ocx-cl10-export-transaction-${process.pid}-${Math.random().toString(16).slice(2)}`); + mkdirSync(home, { recursive: true, mode: 0o700 }); + homes.push(home); + return home; +} + +function syntheticPassResult(caseRecord: CaseRecord) { + return { + scenarioId: caseRecord.id, + suite: caseRecord.suite, + passed: true, + classification: "inconclusive" as const, + assertionResults: caseRecord.assertions.map((assertion) => ({ + id: assertion.id, + operator: assertion.operator, + required: assertion.required, + passed: true, + })), + diagnostics: [], + executionContext: resolveProtocolExecutionContext(caseRecord), + startedAt: 1_700_000_000_000, + completedAt: 1_700_000_001_000, + }; +} + +function seedProtocolProjection(home: string): string { + const authority = loadCaseAuthority(); + const scenario = discoverScenarios(authority, ["responses-core"]) + .find((candidate) => candidate.id === "responses-core.protocol.request-shape") + ?? discoverScenarios(authority, ["responses-core"])[0]; + if (!scenario) throw new Error("no responses-core protocol scenario available"); + const store = createArtifactStore(join(home, "lab", "artifacts")); + try { + persistConformanceResult(syntheticPassResult(scenario), scenario, authority, { + configDir: home, + recordedAt: 1_700_000_001_100, + artifactStore: store, + }); + } finally { + store.close(); + } + rebuildLabProjection(home); + const eventId = queryLabObservations( + { layer: "protocol_conformance", scenarioId: scenario.id }, + undefined, + 10, + home, + ).items[0]?.eventId; + if (!eventId) throw new Error("seeded observation missing"); + return eventId; +} + +test("a provenance failure prevents local public export publication", () => { + const home = tempHome(); + const eventId = seedProtocolProjection(home); + const preview = previewLocalPublicEvidence({ eventIds: [eventId] }, home); + const publisher = getOrCreatePublicPublisher(home).publisher; + const unsigned = buildPublicEvidenceBundle({ + records: preview.bundle.records, + artifacts: preview.bundle.artifacts, + createdDayUtc: preview.bundle.createdDayUtc, + publisher, + }); + const exportPath = join(labExportDir(home), `${unsigned.bundleId}.json`); + const originPath = join( + labPublicOriginDir(home), + `origin-${publisher.keyId}-${unsigned.bundleId}.json`, + ); + + // A directory at the marker pathname makes the provenance commit fail closed. + mkdirSync(originPath, { mode: 0o700 }); + expect(() => exportLocalPublicEvidence({ eventIds: [eventId] }, home)).toThrow(/public origin marker/i); + expect(existsSync(exportPath)).toBe(false); +}); diff --git a/tests/lab-public-file-safety.test.ts b/tests/lab-public-file-safety.test.ts new file mode 100644 index 0000000000..6693e54812 --- /dev/null +++ b/tests/lab-public-file-safety.test.ts @@ -0,0 +1,51 @@ +import { afterEach, expect, test } from "bun:test"; +import { linkSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { readPrivateRegularFile } from "../src/lab/public/file-safety"; +import { PublicEvidenceValidationError } from "../src/lab/public/validate"; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function tempRoot(): string { + const root = mkdtempSync(join(tmpdir(), "ocx-cl10-file-safety-")); + roots.push(root); + return root; +} + +test("descriptor-bound private reads reject a symlink even when O_NOFOLLOW is unavailable", () => { + const root = tempRoot(); + const target = join(root, "target.txt"); + const link = join(root, "link.txt"); + writeFileSync(target, "safe-bytes", { mode: 0o600 }); + try { + symlinkSync(target, link, "file"); + } catch (error) { + if (process.platform === "win32" && (error as NodeJS.ErrnoException).code === "EPERM") return; + throw error; + } + + expect(() => readPrivateRegularFile(link, { + maxBytes: 1024, + errorCode: "unsafe_test_file", + errorMessage: "unsafe test file", + })).toThrow(PublicEvidenceValidationError); +}); + +test("descriptor-bound private reads reject a file with an unrelated hard link", () => { + if (process.platform === "win32") return; + const root = tempRoot(); + const target = join(root, "target.txt"); + const alias = join(root, "alias.txt"); + writeFileSync(target, "safe-bytes", { mode: 0o600 }); + linkSync(target, alias); + + expect(() => readPrivateRegularFile(target, { + maxBytes: 1024, + errorCode: "unsafe_test_file", + errorMessage: "unsafe test file", + })).toThrow(PublicEvidenceValidationError); +}); diff --git a/tests/lab-public-final-review-regressions.test.ts b/tests/lab-public-final-review-regressions.test.ts new file mode 100644 index 0000000000..be37657007 --- /dev/null +++ b/tests/lab-public-final-review-regressions.test.ts @@ -0,0 +1,119 @@ +import { afterEach, expect, test } from "bun:test"; +import { existsSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ensureLabDirs, labPublicOriginDir } from "../src/lab/paths"; +import { + importCommunityEvidenceBundle, + listLocalPublicOrigins, + purgeLocalPublicEvidenceCopies, + publicEvidenceId, + signPublicEvidenceBundle, + writePublicEvidenceBundle, + type PublicEvidenceRecordV1, +} from "../src/lab/public"; +import { listValidPublicOriginsForPurge } from "../src/lab/public/origin-purge"; +import { setPublicEvidencePurgeFaultForTests } from "../src/lab/public/purge-test-fault"; + +const roots: string[] = []; + +afterEach(() => { + setPublicEvidencePurgeFaultForTests(null); + for (const root of roots.splice(0)) { + if (existsSync(root)) rmSync(root, { recursive: true, force: true }); + } +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function record(day = "2026-08-12"): PublicEvidenceRecordV1 { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const body = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: day, + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", body), ...body }; +} + +function bundle(config: string, day = "2026-08-12") { + return signPublicEvidenceBundle({ + records: [record(day)], + artifacts: [], + createdDayUtc: day, + configDir: config, + }); +} + +test("raw community import restores provenance for an exact own-publisher bundle", () => { + const home = configDir("ocx-cl10-raw-own-import-"); + const own = bundle(home); + expect(listLocalPublicOrigins(home)).toEqual([]); + + importCommunityEvidenceBundle(own, home); + expect(listLocalPublicOrigins(home)).toEqual([{ + publisherKeyId: own.publisher.keyId, + bundleId: own.bundleId, + }]); +}); + +test("purge provenance salvage does not truncate valid markers at the operational quota", () => { + const home = configDir("ocx-cl10-purge-origin-overflow-"); + ensureLabDirs(home); + const dir = labPublicOriginDir(home); + + for (let index = 0; index < 1025; index += 1) { + const publisherKeyId = publicEvidenceId("publisher_key", { index }); + const bundleId = publicEvidenceId("bundle", { index }); + writeFileSync( + join(dir, `origin-${publisherKeyId}-${bundleId}.json`), + JSON.stringify({ schemaVersion: "public_origin_v1", publisherKeyId, bundleId }), + { mode: 0o600 }, + ); + } + + expect(listValidPublicOriginsForPurge(home)).toHaveLength(1025); +}); + +test.skipIf(process.platform === "win32")( + "export purge keeps failing closed on retry until POSIX deletion durability is established", + () => { + const home = configDir("ocx-cl10-export-delete-durability-"); + const own = bundle(home); + writePublicEvidenceBundle(own, home); + + const restoreFault = setPublicEvidencePurgeFaultForTests("export_directory_sync"); + try { + expect(() => purgeLocalPublicEvidenceCopies(home)).toThrow(/directory|sync|durab/i); + // The first attempt already removed the export pathname. A retry must still + // fsync the now-empty directory rather than reporting success without durability. + expect(() => purgeLocalPublicEvidenceCopies(home)).toThrow(/directory|sync|durab/i); + } finally { + restoreFault(); + } + expect(() => purgeLocalPublicEvidenceCopies(home)).not.toThrow(); + }, +); diff --git a/tests/lab-public-lifecycle-hardening.test.ts b/tests/lab-public-lifecycle-hardening.test.ts new file mode 100644 index 0000000000..3c6acca4d5 --- /dev/null +++ b/tests/lab-public-lifecycle-hardening.test.ts @@ -0,0 +1,214 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { randomUUID } from "node:crypto"; +import { + existsSync, + linkSync, + mkdirSync, + mkdtempSync, + readdirSync, + rmSync, + unlinkSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + labCommunityDir, + labExportDir, + labPublicPublisherKeyPath, +} from "../src/lab/paths"; +import { + publishPrivateFileExclusive, + setPrivateFileCommitFaultForTests, +} from "../src/lab/public/private-file"; +import { + createPublicEvidenceRevocation, + importCommunityEvidenceBundle, + importCommunityEvidenceRevocation, + listCommunityEvidence, + listLocalPublicOrigins, + purgeLocalPublicEvidenceCopies, + publicEvidenceId, + recordLocalPublicOrigin, + signPublicEvidenceBundle, + writePublicEvidenceBundle, + type PublicEvidenceRecordV1, +} from "../src/lab/public"; + +const roots: string[] = []; +afterEach(() => { + setPrivateFileCommitFaultForTests(null); + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function fixedRecord(): PublicEvidenceRecordV1 { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const withoutRecordId = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: "2026-08-12", + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId }; +} + +function signedBundle(config: string, day = "2026-08-12") { + return signPublicEvidenceBundle({ + records: [fixedRecord()], + artifacts: [], + createdDayUtc: day, + configDir: config, + }); +} + +function addLivePrivateStages(dir: string, count: number): void { + mkdirSync(dir, { recursive: true, mode: 0o700 }); + for (let index = 0; index < count; index += 1) { + const finalName = `bundle-${String(index).padStart(3, "0")}.json`; + writeFileSync( + join(dir, `.${finalName}.${process.pid}.${randomUUID()}.tmp`), + "stage", + { mode: 0o600 }, + ); + } +} + +describe("CL-10 public lifecycle hardening", () => { + test("exclusive private publication never exposes a partial final file", () => { + const root = configDir("ocx-cl10-atomic-"); + const finalPath = join(root, "object.json"); + const bytes = Buffer.from('{"ok":true}', "utf8"); + + setPrivateFileCommitFaultForTests("before_publish"); + expect(() => publishPrivateFileExclusive(finalPath, bytes)).toThrow(/synthetic.*commit failure/i); + expect(existsSync(finalPath)).toBe(false); + expect(readdirSync(root).filter((name) => name.endsWith(".tmp"))).toEqual([]); + + setPrivateFileCommitFaultForTests(null); + expect(publishPrivateFileExclusive(finalPath, bytes)).toEqual({ created: true }); + expect(existsSync(finalPath)).toBe(true); + }); + + test("private staging files do not consume the bounded community object quota", () => { + const publisher = configDir("ocx-cl10-stage-publisher-"); + const consumer = configDir("ocx-cl10-stage-consumer-"); + addLivePrivateStages(labCommunityDir(consumer), 512); + + const bundle = signedBundle(publisher); + expect(importCommunityEvidenceBundle(bundle, consumer)).toMatchObject({ + created: true, + status: "cryptographically_valid", + bundleId: bundle.bundleId, + }); + expect(listCommunityEvidence(consumer)).toEqual([ + expect.objectContaining({ bundleId: bundle.bundleId, activeRecordCount: 1 }), + ]); + }); + + test("durable origin provenance purges local community copies even after export and key corruption", () => { + const local = configDir("ocx-cl10-origin-local-"); + const thirdParty = configDir("ocx-cl10-origin-third-party-"); + const localBundle = signedBundle(local); + const thirdPartyBundle = signedBundle(thirdParty, "2026-08-13"); + + const localExportPath = writePublicEvidenceBundle(localBundle, local); + recordLocalPublicOrigin({ + publisherKeyId: localBundle.publisher.keyId, + bundleId: localBundle.bundleId, + }, local); + importCommunityEvidenceBundle(localBundle, local); + importCommunityEvidenceBundle(thirdPartyBundle, local); + + const localRevocation = createPublicEvidenceRevocation({ + configDir: local, + targetBundle: localBundle, + issuedDayUtc: "2026-08-13", + reason: "privacy_retraction", + targets: [{ kind: "bundle", id: localBundle.bundleId }], + }); + importCommunityEvidenceRevocation(localRevocation, local); + + expect(listLocalPublicOrigins(local)).toEqual([{ + publisherKeyId: localBundle.publisher.keyId, + bundleId: localBundle.bundleId, + }]); + + writeFileSync(localExportPath, "{", { encoding: "utf8" }); + unlinkSync(labPublicPublisherKeyPath(local)); + + expect(purgeLocalPublicEvidenceCopies(local)).toMatchObject({ + deletedCommunityBundles: 1, + deletedCommunityRevocations: 1, + }); + expect(readdirSync(labExportDir(local))).toEqual([]); + expect(listLocalPublicOrigins(local)).toEqual([]); + expect(listCommunityEvidence(local).map((row) => row.bundleId)).toEqual([thirdPartyBundle.bundleId]); + }); + + test("locally-originated hardlinked community path is removed without deleting its peer", () => { + const local = configDir("ocx-cl10-unsafe-community-purge-"); + const localBundle = signedBundle(local); + writePublicEvidenceBundle(localBundle, local); + recordLocalPublicOrigin({ + publisherKeyId: localBundle.publisher.keyId, + bundleId: localBundle.bundleId, + }, local); + const imported = importCommunityEvidenceBundle(localBundle, local); + const peerPath = join(local, "community-hardlink-witness.json"); + + linkSync(imported.path, peerPath); + + expect(purgeLocalPublicEvidenceCopies(local)).toMatchObject({ + deletedExports: 1, + deletedCommunityBundles: 1, + }); + expect(readdirSync(labExportDir(local))).toEqual([]); + expect(existsSync(imported.path)).toBe(false); + expect(existsSync(peerPath)).toBe(true); + }); + + test("duplicate-key revocation JSON is rejected before persistence", () => { + const publisher = configDir("ocx-cl10-dup-rev-publisher-"); + const consumer = configDir("ocx-cl10-dup-rev-consumer-"); + const bundle = signedBundle(publisher); + importCommunityEvidenceBundle(bundle, consumer); + const revocation = createPublicEvidenceRevocation({ + configDir: publisher, + targetBundle: bundle, + issuedDayUtc: "2026-08-13", + reason: "evidence_invalidated", + targets: [{ kind: "record", id: bundle.records[0]!.recordId }], + }); + const raw = JSON.stringify(revocation).replace( + '"schemaVersion":"public_evidence_revocation_v1"', + '"schemaVersion":"public_evidence_revocation_v1","schemaVersion":"public_evidence_revocation_v1"', + ); + + expect(() => importCommunityEvidenceRevocation(raw, consumer)).toThrow(/duplicate json object key/i); + expect(readdirSync(labCommunityDir(consumer)).filter((name) => name.startsWith("revocation-"))).toEqual([]); + }); +}); diff --git a/tests/lab-public-privacy-ipv6.test.ts b/tests/lab-public-privacy-ipv6.test.ts new file mode 100644 index 0000000000..1b8da1dae7 --- /dev/null +++ b/tests/lab-public-privacy-ipv6.test.ts @@ -0,0 +1,22 @@ +import { expect, test } from "bun:test"; +import { + validatePublicEvidencePrivacy, + type PublicEvidenceBundleUnsignedV1, +} from "../src/lab/public"; + +test("public artifact privacy rejects embedded unbracketed IPv6 literals", () => { + const bytes = Buffer.from("artifact 2001:db8::1 content", "utf8"); + const bundle = { + createdDayUtc: "2026-08-13", + records: [], + artifacts: [{ + artifactId: "0".repeat(64), + artifactClass: "verifier_summary", + mediaType: "text/plain", + byteCount: bytes.byteLength, + contentBase64: bytes.toString("base64"), + }], + } as unknown as PublicEvidenceBundleUnsignedV1; + + expect(() => validatePublicEvidencePrivacy(bundle)).toThrow(/IP address|privacy/i); +}); diff --git a/tests/lab-public-provenance-recovery.test.ts b/tests/lab-public-provenance-recovery.test.ts new file mode 100644 index 0000000000..fab186c4ec --- /dev/null +++ b/tests/lab-public-provenance-recovery.test.ts @@ -0,0 +1,196 @@ +import { afterEach, expect, test } from "bun:test"; +import { existsSync, mkdtempSync, readdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { labExportDir, labPublicOriginDir } from "../src/lab/paths"; +import { + createPublicEvidenceRevocation, + importCommunityEvidenceBundle, + importCommunityEvidenceValue, + listCommunityEvidence, + listLocalPublicOrigins, + purgeLocalPublicEvidenceCopies, + publicEvidenceId, + recordLocalPublicOrigin, + signPublicEvidenceBundle, + writePublicEvidenceBundle, + type PublicEvidenceRecordV1, +} from "../src/lab/public"; +import { setPrivateFileCommitFaultForTests } from "../src/lab/public/private-file"; + +const roots: string[] = []; +afterEach(() => { + setPrivateFileCommitFaultForTests(null); + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function fixedRecord(day = "2026-08-12"): PublicEvidenceRecordV1 { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const body = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: day, + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", body), ...body }; +} + +function signedBundle(config: string, day = "2026-08-12") { + return signPublicEvidenceBundle({ + records: [fixedRecord(day)], + artifacts: [], + createdDayUtc: day, + configDir: config, + }); +} + +test("one corrupt origin marker does not discard later valid purge provenance", () => { + const home = configDir("ocx-cl10-origin-salvage-"); + const publisherA = configDir("ocx-cl10-origin-publisher-a-"); + const publisherB = configDir("ocx-cl10-origin-publisher-b-"); + const bundles = [signedBundle(publisherA), signedBundle(publisherB, "2026-08-13")]; + + for (const bundle of bundles) { + importCommunityEvidenceBundle(bundle, home); + recordLocalPublicOrigin({ publisherKeyId: bundle.publisher.keyId, bundleId: bundle.bundleId }, home); + } + + const dir = labPublicOriginDir(home); + const names = readdirSync(dir).sort(); + expect(names).toHaveLength(2); + writeFileSync(join(dir, names[0]!), "{", { mode: 0o600 }); + const validName = names[1]!; + const validBundle = bundles.find((bundle) => validName.includes(bundle.bundleId)); + if (!validBundle) throw new Error(`no bundle matches origin marker ${validName}`); + + expect(() => purgeLocalPublicEvidenceCopies(home)).toThrow(/origin.*classification.*incomplete/i); + expect(listCommunityEvidence(home).map((row) => row.bundleId)).not.toContain(validBundle.bundleId); +}); + +test("origin pressure preserves provenance while the matching local export exists", () => { + const home = configDir("ocx-cl10-origin-export-retain-"); + const bundle = signedBundle(home); + writePublicEvidenceBundle(bundle, home); + recordLocalPublicOrigin({ publisherKeyId: bundle.publisher.keyId, bundleId: bundle.bundleId }, home); + + const dir = labPublicOriginDir(home); + for (let index = 0; index < 1023; index += 1) { + const publisherKeyId = publicEvidenceId("publisher_key", { seed: `old-publisher-${index}` }); + const bundleId = publicEvidenceId("bundle", { seed: `old-bundle-${index}` }); + writeFileSync(join(dir, `origin-${publisherKeyId}-${bundleId}.json`), "{}", { mode: 0o600 }); + } + + const next = { + publisherKeyId: publicEvidenceId("publisher_key", { seed: "next-publisher" }), + bundleId: publicEvidenceId("bundle", { seed: "next-bundle" }), + }; + recordLocalPublicOrigin(next, home); + + expect(existsSync(join(dir, `origin-${bundle.publisher.keyId}-${bundle.bundleId}.json`))).toBe(true); +}); + +test("operator import of an own verified bundle restores missing local-origin provenance", () => { + const home = configDir("ocx-cl10-origin-rehydrate-"); + const bundle = signedBundle(home); + expect(listLocalPublicOrigins(home)).toEqual([]); + + importCommunityEvidenceValue(bundle, home); + expect(listLocalPublicOrigins(home)).toEqual([{ + publisherKeyId: bundle.publisher.keyId, + bundleId: bundle.bundleId, + }]); +}); + +test("operator import of a third-party verified bundle does not create local-origin provenance", () => { + const home = configDir("ocx-cl10-origin-third-party-home-"); + const publisher = configDir("ocx-cl10-origin-third-party-publisher-"); + signedBundle(home); + const bundle = signedBundle(publisher); + + importCommunityEvidenceValue(bundle, home); + expect(listLocalPublicOrigins(home)).toEqual([]); +}); + +test("failed own-origin commit rolls back a newly imported community copy", () => { + const home = configDir("ocx-cl10-origin-rollback-"); + const bundle = signedBundle(home); + const dir = labPublicOriginDir(home); + const exportDir = labExportDir(home); + for (let index = 0; index < 1024; index += 1) { + const publisherKeyId = publicEvidenceId("publisher_key", { seed: `occupied-publisher-${index}` }); + const bundleId = publicEvidenceId("bundle", { seed: `occupied-bundle-${index}` }); + writeFileSync( + join(dir, `origin-${publisherKeyId}-${bundleId}.json`), + JSON.stringify({ schemaVersion: "public_origin_v1", publisherKeyId, bundleId }), + { mode: 0o600 }, + ); + writeFileSync(join(exportDir, `${bundleId}.json`), "retained", { mode: 0o600 }); + } + + expect(() => importCommunityEvidenceValue(bundle, home)).toThrow(/origin marker bound/i); + expect(listCommunityEvidence(home)).toEqual([]); +}); + +test.skipIf(process.platform === "win32")( + "origin and community persistence recover after same-process parent-directory sync failures", + () => { + const home = configDir("ocx-cl10-origin-recovery-"); + const publisher = configDir("ocx-cl10-community-recovery-publisher-"); + const identity = { + publisherKeyId: publicEvidenceId("publisher_key", { seed: "recovery-publisher" }), + bundleId: publicEvidenceId("bundle", { seed: "recovery-bundle" }), + }; + + setPrivateFileCommitFaultForTests("parent_directory_sync"); + expect(() => recordLocalPublicOrigin(identity, home)).toThrow(); + setPrivateFileCommitFaultForTests(null); + expect(() => recordLocalPublicOrigin(identity, home)).not.toThrow(); + + const bundle = signedBundle(publisher); + setPrivateFileCommitFaultForTests("parent_directory_sync"); + expect(() => importCommunityEvidenceBundle(bundle, home)).toThrow(); + setPrivateFileCommitFaultForTests(null); + expect(importCommunityEvidenceBundle(bundle, home)).toMatchObject({ created: false, bundleId: bundle.bundleId }); + }, +); + +test("V1 revocations are bounded to one already-verified anchor bundle", () => { + const publisher = configDir("ocx-cl10-revocation-anchor-"); + const first = signedBundle(publisher, "2026-08-12"); + const second = signedBundle(publisher, "2026-08-13"); + + expect(() => createPublicEvidenceRevocation({ + configDir: publisher, + targetBundle: first, + issuedDayUtc: "2026-08-13", + reason: "superseded", + targets: [ + { kind: "bundle", id: first.bundleId }, + { kind: "bundle", id: second.bundleId }, + ], + })).toThrow(/target|unknown/i); +}); diff --git a/tests/lab-public-review-fixes.test.ts b/tests/lab-public-review-fixes.test.ts new file mode 100644 index 0000000000..82c9c701c7 --- /dev/null +++ b/tests/lab-public-review-fixes.test.ts @@ -0,0 +1,261 @@ +import { afterEach, expect, test } from "bun:test"; +import { + existsSync, + mkdtempSync, + readdirSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ensureLabDirs, labPublicOriginDir } from "../src/lab/paths"; +import { purgeSensitiveEvidence } from "../src/lab/ledger/purge"; +import { replayLabLedger } from "../src/lab/ledger/store"; +import * as publicApi from "../src/lab/public"; +import { setPublicEvidencePurgeFaultForTests } from "../src/lab/public/purge-test-fault"; +import { + importCommunityEvidenceBundle, + importCommunityEvidenceRevocation, + purgeLocalPublicEvidenceCopies, + publicEvidenceId, + recordLocalPublicOrigin, + signPublicEvidenceBundle, + writePublicEvidenceBundle, + PublicEvidenceValidationError, + type PublicEvidenceRecordV1, +} from "../src/lab/public"; + +const roots: string[] = []; +afterEach(() => { + setPublicEvidencePurgeFaultForTests(null); + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function hex(seed: string): string { + return Bun.CryptoHasher.hash("sha256", seed, "hex"); +} + +function fixedRecord(): PublicEvidenceRecordV1 { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const withoutRecordId = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: "2026-08-12", + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId }; +} + +function signedBundle(config: string) { + return signPublicEvidenceBundle({ + records: [fixedRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + configDir: config, + }); +} + +test("decoded community objects are depth-bounded before JCS canonicalization", () => { + const consumer = configDir("ocx-cl10-object-bound-"); + let raw: unknown = { leaf: true }; + for (let index = 0; index < 20_000; index += 1) raw = { nested: raw }; + + try { + importCommunityEvidenceBundle(raw, consumer); + throw new Error("expected bounded object rejection"); + } catch (error) { + expect(error).toBeInstanceOf(PublicEvidenceValidationError); + expect((error as PublicEvidenceValidationError).code).toBe("community_depth"); + } +}); + +test("public barrel does not expose private test fault setters", () => { + expect("setPrivateFileCommitFaultForTests" in publicApi).toBe(false); + expect("setPublicEvidencePurgeFaultForTests" in publicApi).toBe(false); +}); + +test("foreign origin entries do not consume marker quota but remain explicitly unsafe to list", () => { + const home = configDir("ocx-cl10-origin-bound-"); + ensureLabDirs(home); + const dir = labPublicOriginDir(home); + for (let index = 0; index < 1024; index += 1) { + writeFileSync(join(dir, `occupied-${String(index).padStart(4, "0")}`), "x", { mode: 0o600 }); + } + + const current = { + publisherKeyId: hex("publisher-bound"), + bundleId: hex("bundle-bound"), + }; + expect(() => recordLocalPublicOrigin(current, home)).not.toThrow(); + expect(readdirSync(dir)).toHaveLength(1025); + expect(() => publicApi.listLocalPublicOrigins(home)).toThrow(/unexpected public origin marker entry/i); +}); + +test("public origin pressure reclaims markers with no community copy", () => { + const home = configDir("ocx-cl10-origin-reclaim-"); + ensureLabDirs(home); + const dir = labPublicOriginDir(home); + for (let index = 0; index < 1024; index += 1) { + const publisherKeyId = hex(`publisher-old-${index}`); + const bundleId = hex(`bundle-old-${index}`); + writeFileSync( + join(dir, `origin-${publisherKeyId}-${bundleId}.json`), + "{}", + { mode: 0o600 }, + ); + } + + const current = { publisherKeyId: hex("publisher-current"), bundleId: hex("bundle-current") }; + recordLocalPublicOrigin(current, home); + const names = readdirSync(dir); + expect(names).toHaveLength(1); + expect(names[0]).toBe(`origin-${current.publisherKeyId}-${current.bundleId}.json`); +}); + +test("corrupt origin provenance cannot retain mandatory local export bytes and reports incomplete classification", () => { + const home = configDir("ocx-cl10-origin-corrupt-"); + const bundle = signedBundle(home); + writePublicEvidenceBundle(bundle, home); + recordLocalPublicOrigin({ publisherKeyId: bundle.publisher.keyId, bundleId: bundle.bundleId }, home); + const originEntry = readdirSync(labPublicOriginDir(home))[0]!; + writeFileSync(join(labPublicOriginDir(home), originEntry), "{", { mode: 0o600 }); + + let failure: unknown; + try { + purgeLocalPublicEvidenceCopies(home); + } catch (error) { + failure = error; + } + expect(failure).toBeInstanceOf(PublicEvidenceValidationError); + expect((failure as PublicEvidenceValidationError).code).toBe("public_origin_incomplete"); + expect(readdirSync(ensureLabDirs(home).exportDir)).toEqual([]); +}); + +test("unsafe locally-originated community copies are removed without blocking sensitive export purge", () => { + const home = configDir("ocx-cl10-community-unsafe-"); + const bundle = signedBundle(home); + writePublicEvidenceBundle(bundle, home); + recordLocalPublicOrigin({ publisherKeyId: bundle.publisher.keyId, bundleId: bundle.bundleId }, home); + const imported = importCommunityEvidenceBundle(bundle, home); + writeFileSync(imported.path, Buffer.alloc(2 * 1024 * 1024 + 1, 0x78), { mode: 0o600 }); + + const result = purgeLocalPublicEvidenceCopies(home); + expect(result.deletedExports).toBe(1); + expect(result.deletedCommunityBundles).toBe(1); + expect(readdirSync(ensureLabDirs(home).exportDir)).toEqual([]); + expect(readdirSync(labPublicOriginDir(home))).toEqual([]); + expect(existsSync(imported.path)).toBe(false); +}); + +test("missing direct revocation bundle target reports stable revocation_target error", () => { + const home = configDir("ocx-cl10-missing-revocation-target-"); + ensureLabDirs(home); + + let failure: unknown; + try { + importCommunityEvidenceRevocation({ + publisher: { keyId: hex("missing-publisher") }, + targets: [{ kind: "bundle", id: hex("missing-bundle") }], + }, home); + } catch (error) { + failure = error; + } + + expect(failure).toBeInstanceOf(PublicEvidenceValidationError); + expect((failure as PublicEvidenceValidationError).code).toBe("revocation_target"); + expect((failure as Error).message).toBe("revocation target bundle not found"); +}); + +test("failed export purge is omitted from the durable tombstone action set", () => { + const home = configDir("ocx-cl10-tombstone-export-"); + const paths = ensureLabDirs(home); + writeFileSync(join(paths.scratchDir, "scratch.txt"), "scratch", { mode: 0o600 }); + writeFileSync(join(paths.exportDir, "sensitive.txt"), "sensitive", { mode: 0o600 }); + const restoreFault = setPublicEvidencePurgeFaultForTests("before_export_delete"); + + let failure: unknown; + try { + purgeSensitiveEvidence({ + configDir: home, + purgeActions: ["export", "scratch"], + recordedAt: Date.UTC(2026, 7, 13, 6, 0, 0), + }); + } catch (error) { + failure = error; + } finally { + restoreFault(); + } + expect(failure).toBeInstanceOf(Error); + + const tombstones = replayLabLedger(paths.ledgerPath).events.filter((event) => event.eventKind === "purge_tombstone"); + expect(tombstones).toHaveLength(1); + expect(tombstones[0]!.purgeActions).toEqual(["scratch"]); +}); + +test("failed export plus ledger does not persist a targetless tombstone", () => { + const home = configDir("ocx-cl10-tombstone-export-ledger-"); + const paths = ensureLabDirs(home); + const restoreFault = setPublicEvidencePurgeFaultForTests("before_export_delete"); + let failure: unknown; + try { + purgeSensitiveEvidence({ + configDir: home, + purgeActions: ["export", "ledger"], + recordedAt: Date.UTC(2026, 7, 14, 6, 0, 0), + }); + } catch (error) { + failure = error; + } finally { + restoreFault(); + } + + expect(failure).toBeInstanceOf(Error); + expect(replayLabLedger(paths.ledgerPath).events).toEqual([]); +}); + +test("failed export plus sqlite still rebuilds projection from the unchanged ledger", () => { + const home = configDir("ocx-cl10-tombstone-export-sqlite-"); + const paths = ensureLabDirs(home); + expect(existsSync(paths.sqlitePath)).toBe(false); + const restoreFault = setPublicEvidencePurgeFaultForTests("before_export_delete"); + let failure: unknown; + try { + purgeSensitiveEvidence({ + configDir: home, + purgeActions: ["export", "sqlite"], + recordedAt: Date.UTC(2026, 7, 14, 6, 5, 0), + }); + } catch (error) { + failure = error; + } finally { + restoreFault(); + } + + expect(failure).toBeInstanceOf(Error); + expect(replayLabLedger(paths.ledgerPath).events).toEqual([]); + expect(existsSync(paths.sqlitePath)).toBe(true); +}); diff --git a/tests/lab-public-route-registry.test.ts b/tests/lab-public-route-registry.test.ts new file mode 100644 index 0000000000..e13b8fdb6c --- /dev/null +++ b/tests/lab-public-route-registry.test.ts @@ -0,0 +1,20 @@ +import { describe, expect, test } from "bun:test"; +import { PUBLIC_ROUTE_REGISTRY_V1, validatePublicRouteRegistryManifest } from "../src/lab/public"; + +const REVIEWED_AUTHORITY_SOURCE_COMMIT = "75a21417657ba5a3033198be0d8ae949de723d11"; + +describe("CL-10 public route registry authority", () => { + test("pins the reviewed OpenAI gpt-5.6-sol authority exactly", () => { + const manifest = validatePublicRouteRegistryManifest(PUBLIC_ROUTE_REGISTRY_V1); + + expect(manifest.registryVersion).toBe("2026-08-13.v2"); + expect(manifest.sourceCommit).toBe(REVIEWED_AUTHORITY_SOURCE_COMMIT); + expect(manifest.entries).toEqual([ + { + providerId: "openai", + modelId: "gpt-5.6-sol", + adapterFamilies: ["openai-responses"], + }, + ]); + }); +}); diff --git a/tests/lab-public-security-regressions.test.ts b/tests/lab-public-security-regressions.test.ts new file mode 100644 index 0000000000..ee86428655 --- /dev/null +++ b/tests/lab-public-security-regressions.test.ts @@ -0,0 +1,188 @@ +import { afterEach, expect, test } from "bun:test"; +import { existsSync, mkdtempSync, rmSync, statSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { jcsStringify } from "../src/lab/conformance/jcs"; +import type { ObservationEvent } from "../src/lab/events/types"; +import { labPublicPublisherKeyPath } from "../src/lab/paths"; +import { publishPrivateFileExclusive } from "../src/lab/public/private-file"; +import { validatePublicEvidencePrivacy } from "../src/lab/public/privacy"; +import { projectPublicEvidenceRecord } from "../src/lab/public/project"; +import { getOrCreatePublicPublisher } from "../src/lab/public/signature"; +import type { PublicEvidenceBundleUnsignedV1 } from "../src/lab/public/types"; +import { + resetHardenedStateForTests, + setIcaclsRunnerForTests, + setPlatformForTests, +} from "../src/lib/windows-secret-acl"; + +const roots: string[] = []; + +afterEach(() => { + setIcaclsRunnerForTests(null); + setPlatformForTests(null); + resetHardenedStateForTests(); + for (const root of roots.splice(0)) { + if (existsSync(root)) rmSync(root, { recursive: true, force: true }); + } +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +test("JCS rejects sparse JavaScript arrays instead of collapsing holes", () => { + const sparse = new Array<unknown>(1); + expect(() => jcsStringify(sparse)).toThrow(/sparse|array hole/i); +}); + +test("JCS rejects non-plain objects instead of collapsing canonical identity", () => { + const values: unknown[] = [ + new Date(0), + new Map([["a", 1]]), + new Set([1]), + new Uint8Array([1, 2, 3]), + ]; + for (const value of values) { + expect(() => jcsStringify(value)).toThrow(/plain JSON object/i); + } +}); + +test("public projection maps JCS-invalid public fields to not_exportable", () => { + const observation = { + evidenceLayer: "protocol_conformance", + subject: { + subjectKind: "protocol", + effectiveAdapter: "openai-chat", + opencodexCompatibilityVersion: "2.13.0", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-\uD800", + }, + } as unknown as ObservationEvent; + + expect(projectPublicEvidenceRecord({ observation, verdict: "VERIFIED" })).toEqual({ + status: "not_exportable", + reason: "unsafe_public_field", + }); +}); + +test("public projection drops invalid completion timestamps with a diagnostic code", () => { + const observation = { + evidenceLayer: "protocol_conformance", + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + completedAt: Date.UTC(10_000, 0, 1), + assertions: [], + subject: { + subjectKind: "protocol", + effectiveAdapter: "openai-chat", + opencodexCompatibilityVersion: "2.13.0", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }, + } as unknown as ObservationEvent; + + expect(projectPublicEvidenceRecord({ observation, verdict: "VERIFIED" })).toEqual({ + status: "not_exportable", + reason: "unsafe_public_field", + detailCode: "public_selection_time", + }); +}); + +test("public privacy rejects embedded POSIX absolute paths across common runtime roots", () => { + const localPaths = [ + "/var/folders/9k/opencodex/output.json", + "/dev/shm/opencodex.sock", + "/run/user/1000/opencodex/token", + "/Library/Application Support/opencodex/config.json", + ]; + + for (const localPath of localPaths) { + const bytes = Buffer.from(`diagnostic path=${localPath}`, "utf8"); + const bundle = { + createdDayUtc: "2026-08-14", + records: [], + artifacts: [{ + artifactId: "0".repeat(64), + artifactClass: "verifier_summary", + mediaType: "text/plain", + byteCount: bytes.byteLength, + contentBase64: bytes.toString("base64"), + }], + } as unknown as PublicEvidenceBundleUnsignedV1; + + expect(() => validatePublicEvidencePrivacy(bundle)).toThrow(/local path|privacy/i); + } +}); + +test("private publication prepares an empty stage before writing secret bytes", () => { + const root = configDir("ocx-cl10-private-stage-prepare-"); + const finalPath = join(root, "secret.bin"); + const observedSizes: number[] = []; + + expect(publishPrivateFileExclusive(finalPath, Buffer.from("secret", "utf8"), { + prepareStage: stagePath => observedSizes.push(statSync(stagePath).size), + })).toEqual({ created: true }); + expect(observedSizes).toEqual([0]); +}); + +test("publisher key creation applies required Windows secret ACL hardening to the final key path", () => { + const home = configDir("ocx-cl10-windows-publisher-acl-"); + const keyPath = labPublicPublisherKeyPath(home); + const calls: string[][] = []; + + resetHardenedStateForTests(); + setPlatformForTests("win32"); + setIcaclsRunnerForTests((args) => { + calls.push(args); + return { success: true, exitCode: 0, timedOut: false, stdout: "" }; + }); + + expect(getOrCreatePublicPublisher(home).publisher.algorithm).toBe("ed25519"); + expect(existsSync(keyPath)).toBe(true); + expect(calls.some((args) => args[0] === keyPath && args.includes("/grant:r"))).toBe(true); + expect(calls.some((args) => args[0] === keyPath && args.includes("/inheritance:r"))).toBe(true); +}); + +test("publisher key creation never publishes the final path when required Windows ACL hardening fails", () => { + const home = configDir("ocx-cl10-windows-publisher-acl-fail-"); + const keyPath = labPublicPublisherKeyPath(home); + + resetHardenedStateForTests(); + setPlatformForTests("win32"); + setIcaclsRunnerForTests(() => ({ + success: false, + exitCode: 5, + timedOut: false, + stdout: "", + })); + + expect(() => getOrCreatePublicPublisher(home)).toThrow(/ACL hardening/i); + expect(existsSync(keyPath)).toBe(false); +}); + +test("publisher key ACL failures preserve their underlying cause", () => { + const home = configDir("ocx-cl10-windows-publisher-acl-cause-"); + + resetHardenedStateForTests(); + setPlatformForTests("win32"); + setIcaclsRunnerForTests(() => { + throw new Error("synthetic icacls runner failure"); + }); + + let caught: unknown; + try { + getOrCreatePublicPublisher(home); + } catch (error) { + caught = error; + } + + expect(caught).toBeInstanceOf(Error); + expect((caught as Error & { cause?: unknown }).cause).toBeInstanceOf(Error); +}); diff --git a/tests/lab-public-surfaces.test.ts b/tests/lab-public-surfaces.test.ts new file mode 100644 index 0000000000..404112db45 --- /dev/null +++ b/tests/lab-public-surfaces.test.ts @@ -0,0 +1,327 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { existsSync, mkdirSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { handleLabCommand } from "../src/cli/lab"; +import { + labExportDir, + labPublicPublisherKeyPath, + persistConformanceResult, + rebuildLabProjection, +} from "../src/lab"; +import { labCommunityDir } from "../src/lab/paths"; +import { createArtifactStore } from "../src/lab/artifacts/store"; +import { resolveProtocolExecutionContext } from "../src/lab/conformance/executor"; +import { discoverScenarios, loadCaseAuthority } from "../src/lab/conformance/manifest"; +import type { CaseRecord } from "../src/lab/conformance/types"; +import { queryLabObservations } from "../src/lab/query"; +import { handleManagementAPI } from "../src/server/management-api"; +import type { OcxConfig } from "../src/types"; +import { ManagementRequest } from "./helpers/management-auth"; + +const HOMES: string[] = []; + +afterEach(() => { + for (const home of HOMES.splice(0)) rmSync(home, { recursive: true, force: true }); + delete process.env.OPENCODEX_HOME; +}); + +function tempHome(): string { + const home = join(tmpdir(), `ocx-cl10-surfaces-${process.pid}-${Math.random().toString(16).slice(2)}`); + mkdirSync(home, { recursive: true, mode: 0o700 }); + HOMES.push(home); + return home; +} + +function syntheticPassResult(caseRecord: CaseRecord) { + return { + scenarioId: caseRecord.id, + suite: caseRecord.suite, + passed: true, + classification: "inconclusive" as const, + assertionResults: caseRecord.assertions.map((assertion) => ({ + id: assertion.id, + operator: assertion.operator, + required: assertion.required, + passed: true, + observedSummary: "PRIVATE-CANARY-OBSERVED", + })), + diagnostics: ["PRIVATE-CANARY-DIAGNOSTIC"], + executionContext: resolveProtocolExecutionContext(caseRecord), + startedAt: 1_700_000_000_000, + completedAt: 1_700_000_001_000, + }; +} + +function seedProtocolProjection(home: string): string { + const authority = loadCaseAuthority(); + const scenario = discoverScenarios(authority, ["responses-core"]) + .find((candidate) => candidate.id === "responses-core.protocol.request-shape") + ?? discoverScenarios(authority, ["responses-core"])[0]; + if (!scenario) throw new Error("no responses-core protocol scenario available"); + const store = createArtifactStore(join(home, "lab", "artifacts")); + try { + persistConformanceResult(syntheticPassResult(scenario), scenario, authority, { + configDir: home, + recordedAt: 1_700_000_001_100, + artifactStore: store, + }); + } finally { + store.close(); + } + rebuildLabProjection(home); + const rows = queryLabObservations( + { layer: "protocol_conformance", scenarioId: scenario.id }, + undefined, + 10, + home, + ); + const eventId = rows.items[0]?.eventId; + if (!eventId) throw new Error("seeded observation missing"); + return eventId; +} + +function config(home: string): OcxConfig { + void home; + return { port: 0, defaultProvider: "openai-apikey", providers: {} } as OcxConfig; +} + +async function api( + home: string, + path: string, + init: { method?: string; body?: unknown } = {}, +): Promise<Response> { + process.env.OPENCODEX_HOME = home; + const req = new ManagementRequest(`http://127.0.0.1${path}`, { + method: init.method ?? "GET", + ...(init.body !== undefined + ? { headers: { "content-type": "application/json" }, body: JSON.stringify(init.body) } + : {}), + }); + const response = await handleManagementAPI(req, new URL(req.url), config(home), { + refreshCodexCatalog: async () => {}, + }); + expect(response).not.toBeNull(); + return response!; +} + +async function captureCli(argv: string[], home: string): Promise<{ code: number; stdout: string; stderr: string }> { + const stdout: string[] = []; + const stderr: string[] = []; + const originalLog = console.log; + const originalError = console.error; + console.log = (...args: unknown[]) => { stdout.push(args.join(" ")); }; + console.error = (...args: unknown[]) => { stderr.push(args.join(" ")); }; + try { + return { + code: await handleLabCommand(argv, { configDir: home }), + stdout: stdout.join("\n"), + stderr: stderr.join("\n"), + }; + } finally { + console.log = originalLog; + console.error = originalError; + } +} + +function installNetworkCanary(): () => void { + const original = globalThis.fetch; + globalThis.fetch = (async () => { + throw new Error("CL10-NETWORK-CANARY"); + }) as typeof fetch; + return () => { globalThis.fetch = original; }; +} + +describe("CL-10 CLI local public evidence", () => { + test("preview is network-free, identifier-safe, and does not create publisher or export state", async () => { + const home = tempHome(); + const eventId = seedProtocolProjection(home); + const unknownEventId = "0".repeat(64); + const restoreFetch = installNetworkCanary(); + try { + const result = await captureCli([ + "public", "preview", "--event", eventId, "--event", unknownEventId, "--json", + ], home); + expect(result.code).toBe(0); + const body = JSON.parse(result.stdout) as { + bundle: { records: unknown[]; publisher?: unknown }; + excluded: Array<{ selectionIndex: number; reason: string; eventId?: string }>; + }; + expect(body.bundle.records).toHaveLength(1); + expect(body.bundle).not.toHaveProperty("publisher"); + expect(body.excluded).toEqual([{ selectionIndex: 1, reason: "event_not_found" }]); + expect(body.excluded[0]).not.toHaveProperty("eventId"); + expect(result.stdout).not.toContain(eventId); + expect(result.stdout).not.toContain(unknownEventId); + expect(existsSync(labPublicPublisherKeyPath(home))).toBe(false); + expect(existsSync(labExportDir(home)) ? readdirSync(labExportDir(home)) : []).toEqual([]); + expect(result.stdout).not.toContain("PRIVATE-CANARY"); + } finally { + restoreFetch(); + } + }); + + test("explicit export signs and stores, then verify/import/community remain local", async () => { + const home = tempHome(); + const eventId = seedProtocolProjection(home); + const restoreFetch = installNetworkCanary(); + try { + const exported = await captureCli(["public", "export", "--event", eventId, "--json"], home); + expect(exported.code).toBe(0); + const exportBody = JSON.parse(exported.stdout) as { + bundle: { bundleId: string; publisher: { keyId: string } }; + stored: { path: string; created: boolean }; + }; + expect(exportBody.bundle.publisher.keyId).toMatch(/^[0-9a-f]{64}$/); + expect(exportBody.stored).toEqual({ path: "<private>", created: true }); + expect(exported.stdout).not.toContain(home); + const privateExportPath = join(labExportDir(home), `${exportBody.bundle.bundleId}.json`); + expect(existsSync(privateExportPath)).toBe(true); + + const verified = await captureCli(["public", "verify", "--file", privateExportPath, "--json"], home); + expect(verified.code).toBe(0); + expect(JSON.parse(verified.stdout)).toMatchObject({ + status: "cryptographically_valid", + bundleId: exportBody.bundle.bundleId, + publisherKeyId: exportBody.bundle.publisher.keyId, + locallyVerified: false, + }); + + const ledgerBefore = readFileSync(join(home, "lab", "compatibility.jsonl")); + const sqliteBefore = readFileSync(join(home, "lab", "compatibility.sqlite")); + const imported = await captureCli(["public", "import", "--file", privateExportPath, "--json"], home); + expect(imported.code).toBe(0); + const importedBody = JSON.parse(imported.stdout) as Record<string, unknown>; + expect(importedBody).toMatchObject({ + status: "cryptographically_valid", + trustClass: "community_untrusted_v1", + bundleId: exportBody.bundle.bundleId, + }); + expect(importedBody).not.toHaveProperty("path"); + expect(imported.stdout).not.toContain(home); + expect(readFileSync(join(home, "lab", "compatibility.jsonl")).equals(ledgerBefore)).toBe(true); + expect(readFileSync(join(home, "lab", "compatibility.sqlite")).equals(sqliteBefore)).toBe(true); + + const community = await captureCli(["public", "community", "--json"], home); + expect(community.code).toBe(0); + const communityBody = JSON.parse(community.stdout) as { evidence: Array<{ bundleId: string; trustClass: string }> }; + expect(communityBody.evidence).toEqual([ + expect.objectContaining({ bundleId: exportBody.bundle.bundleId, trustClass: "community_untrusted_v1" }), + ]); + } finally { + restoreFetch(); + } + }); + + test("has no publish command", async () => { + const home = tempHome(); + const result = await captureCli(["public", "publish", "--json"], home); + expect(result.code).toBe(2); + expect(result.stderr).toMatch(/unknown public subcommand|usage/i); + }); +}); + +describe("CL-10 management local public evidence", () => { + test("preview/export/verify/import/community are explicit authenticated local actions", async () => { + const home = tempHome(); + const eventId = seedProtocolProjection(home); + const restoreFetch = installNetworkCanary(); + try { + const preview = await api(home, "/api/lab/public/preview", { + method: "POST", + body: { eventIds: [eventId] }, + }); + expect(preview.status).toBe(200); + const previewBody = await preview.json() as { bundle: { records: unknown[]; publisher?: unknown } }; + expect(previewBody.bundle.records).toHaveLength(1); + expect(previewBody.bundle).not.toHaveProperty("publisher"); + expect(existsSync(labPublicPublisherKeyPath(home))).toBe(false); + + const exported = await api(home, "/api/lab/public/export", { + method: "POST", + body: { eventIds: [eventId] }, + }); + expect(exported.status).toBe(200); + const exportBody = await exported.json() as { + bundle: { bundleId: string; publisher: { keyId: string } }; + stored: { path: string; created: boolean }; + }; + expect(exportBody.stored).toEqual({ path: "<private>", created: true }); + expect(JSON.stringify(exportBody)).not.toContain(home); + + const verified = await api(home, "/api/lab/public/verify", { + method: "POST", + body: { bundle: exportBody.bundle }, + }); + expect(verified.status).toBe(200); + expect(await verified.json()).toMatchObject({ + status: "cryptographically_valid", + bundleId: exportBody.bundle.bundleId, + locallyVerified: false, + }); + + const ledgerBefore = readFileSync(join(home, "lab", "compatibility.jsonl")); + const imported = await api(home, "/api/lab/public/community/import", { + method: "POST", + body: { bundle: exportBody.bundle }, + }); + expect(imported.status).toBe(200); + const importedBody = await imported.json() as Record<string, unknown>; + expect(importedBody).toMatchObject({ + status: "cryptographically_valid", + trustClass: "community_untrusted_v1", + }); + expect(importedBody).not.toHaveProperty("path"); + expect(JSON.stringify(importedBody)).not.toContain(home); + expect(readFileSync(join(home, "lab", "compatibility.jsonl")).equals(ledgerBefore)).toBe(true); + + const community = await api(home, "/api/lab/public/community"); + expect(community.status).toBe(200); + expect(await community.json()).toMatchObject({ + evidence: [expect.objectContaining({ bundleId: exportBody.bundle.bundleId })], + }); + } finally { + restoreFetch(); + } + }); + + test("busy community lock is a prompt retryable service response", async () => { + const home = tempHome(); + const lockPath = join(labCommunityDir(home), ".mutation-lock"); + mkdirSync(lockPath, { recursive: true, mode: 0o700 }); + writeFileSync( + join(lockPath, "owner.json"), + JSON.stringify({ + pid: process.pid, + token: "00000000-0000-4000-8000-000000000000", + createdAt: Date.now(), + }), + { encoding: "utf8", mode: 0o600 }, + ); + + const startedAt = performance.now(); + const response = await api(home, "/api/lab/public/community"); + const elapsedMs = performance.now() - startedAt; + + expect(elapsedMs).toBeLessThan(500); + expect(response.status).toBe(503); + expect(response.headers.get("retry-after")).toBe("1"); + expect(await response.json()).toMatchObject({ + error: { code: "community_cache_busy" }, + }); + }); + + test("does not expose a remote publish endpoint", async () => { + const home = tempHome(); + process.env.OPENCODEX_HOME = home; + const req = new ManagementRequest("http://127.0.0.1/api/lab/public/publish", { + method: "POST", + headers: { "content-type": "application/json" }, + body: "{}", + }); + const res = await handleManagementAPI(req, new URL(req.url), config(home), { + refreshCodexCatalog: async () => {}, + }); + expect(res).toBeNull(); + }); +}); diff --git a/tests/lab-public-wire-contract.test.ts b/tests/lab-public-wire-contract.test.ts new file mode 100644 index 0000000000..6ed82e8268 --- /dev/null +++ b/tests/lab-public-wire-contract.test.ts @@ -0,0 +1,132 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { labPublicPublisherKeyPath } from "../src/lab/paths"; +import { + buildPublicEvidenceBundle, + importCommunityEvidenceBundle, + parseStrictPublicJson, + publicEvidenceId, + signPublicEvidenceBundle, + verifyPublicEvidenceBundle, +} from "../src/lab/public"; + +// Deterministic test-only key material is assembled at runtime so leak scanners do not +// mistake the fixture for a deployable private-key credential. +const FIXED_PRIVATE_KEY = [ + `-----BEGIN PRIVATE ${"KEY"}-----`, + ["MC4CAQAwBQYDK2VwBCIEIAABAgMEBQYH", "CAkKCwwNDg8QERITFBUWFxgZGhscHR4f"].join(""), + `-----END PRIVATE ${"KEY"}-----`, + "", +].join("\n"); +const FIXED_PUBLIC_KEY = "MCowBQYDK2VwAyEAA6EHv/POEL4dcN0Y50vAmWfk1jCbpQ1fHdyGZBJVMbg="; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function configDir(prefix: string): string { + const root = mkdtempSync(join(tmpdir(), prefix)); + roots.push(root); + return root; +} + +function installFixedPublisherKey(config: string): void { + const path = labPublicPublisherKeyPath(config); + mkdirSync(dirname(path), { recursive: true, mode: 0o700 }); + writeFileSync(path, FIXED_PRIVATE_KEY, { encoding: "utf8", mode: 0o600 }); + if (process.platform !== "win32") chmodSync(path, 0o600); +} + +function fixedRecord() { + const subject = { + subjectKind: "protocol" as const, + compatibilityVersion: "2.13.0", + adapterFamily: "openai-chat" as const, + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + }; + const subjectId = publicEvidenceId("subject", subject); + const withoutRecordId = { + subjectId, + evidenceLayer: "protocol_conformance" as const, + suiteId: "responses-core", + suiteVersion: "1.0.0", + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1.0.0", + verdict: "VERIFIED" as const, + observedDayUtc: "2026-08-12", + subject, + assertions: [ + { id: "method", required: true, passed: true }, + { id: "message", required: true, passed: true }, + { id: "temperature", required: true, passed: true }, + ], + }; + return { recordId: publicEvidenceId("record", withoutRecordId), ...withoutRecordId }; +} + +function fixedBundle(config: string) { + installFixedPublisherKey(config); + return signPublicEvidenceBundle({ + records: [fixedRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + configDir: config, + }); +} + +describe("CL-10 public wire contract", () => { + test("freezes the RFC 8785/domain-separated bundle and Ed25519 signature vector", () => { + const bundle = fixedBundle(configDir("ocx-cl10-wire-publisher-")); + + expect(bundle.publisher.publicKey).toBe(FIXED_PUBLIC_KEY); + expect(bundle.publisher.keyId).toBe("4d5a347afcc7a1ac8d2dd4e573f0fbca2d2e90dd472c35df5c72bf2d2afca08f"); + expect(bundle.records[0]!.subjectId).toBe("982a06b98a218df5ed68ae88f5f203e1911a3e875343c6ed8d5d0b74ff4c2b25"); + expect(bundle.records[0]!.recordId).toBe("5bec20821bbf01f831e74ba469e7f18481c1209fdef209c76f482105de3e406d"); + expect(bundle.bundleId).toBe("a7598b68a4cf884dc381b1d88111e74bfad5e74ceae2be8de55b88bac3250401"); + expect(bundle.bundleDigest).toBe("aeef2f3e64a131588f6a34aaea1172c352a0c803f838690c2d6ee652ca74fb87"); + expect(bundle.signature).toEqual({ + algorithm: "ed25519", + signedDigest: "aeef2f3e64a131588f6a34aaea1172c352a0c803f838690c2d6ee652ca74fb87", + signature: "UAiI7Mz4/yIU5XjSuNZFSuyFPoAvGCy+x9cpTCwYKnFDq20AP6ipV3zowD3S4KP2iYfkXyHTMsMH3CEnz6lCBw==", + }); + expect(verifyPublicEvidenceBundle(bundle)).toEqual({ status: "cryptographically_valid" }); + }); + + test("rejects non-canonical publisher public-key Base64", () => { + const publicKey = `${FIXED_PUBLIC_KEY}\n`; + const publisher = { + algorithm: "ed25519" as const, + publicKey, + keyId: publicEvidenceId("publisher_key", { algorithm: "ed25519", publicKey }), + }; + + expect(() => buildPublicEvidenceBundle({ + records: [fixedRecord()], + artifacts: [], + createdDayUtc: "2026-08-12", + publisher, + })).toThrow(/canonical base64/i); + }); + + test("rejects duplicate JSON object keys before community parsing", () => { + const publisherDir = configDir("ocx-cl10-wire-publisher-"); + const consumerDir = configDir("ocx-cl10-wire-consumer-"); + const bundle = fixedBundle(publisherDir); + const raw = JSON.stringify(bundle).replace( + '"schemaVersion":"public_evidence_bundle_v1"', + '"schemaVersion":"public_evidence_bundle_v1","schemaVersion":"public_evidence_bundle_v1"', + ); + + expect(() => importCommunityEvidenceBundle(raw, consumerDir)).toThrow(/duplicate json object key/i); + }); + + test("rejects public JSON deeper than the V1 import bound before JSON.parse materialization", () => { + const raw = Buffer.from(`${"[".repeat(9)}0${"]".repeat(9)}`, "utf8"); + expect(() => parseStrictPublicJson(raw)).toThrow(/nesting depth exceeds 8/i); + }); +}); diff --git a/tests/lab-read-filter-validation.test.ts b/tests/lab-read-filter-validation.test.ts new file mode 100644 index 0000000000..48d93eaa0f --- /dev/null +++ b/tests/lab-read-filter-validation.test.ts @@ -0,0 +1,29 @@ +import { describe, expect, test } from "bun:test"; +import type { OcxConfig } from "../src/types"; +import { handleManagementAPI } from "../src/server/management-api"; +import { ManagementRequest } from "./helpers/management-auth"; + +const config = { providers: {} } as OcxConfig; + +async function apiGet(path: string): Promise<Response> { + const req = new ManagementRequest(`http://127.0.0.1${path}`, { method: "GET" }); + const response = await handleManagementAPI(req, new URL(req.url), config); + expect(response).not.toBeNull(); + return response!; +} + +describe("Compatibility Lab management read filter validation", () => { + test("rejects invalid excluded values instead of silently dropping the filter", async () => { + const response = await apiGet("/api/lab/events?excluded=maybe"); + expect(response.status).toBe(400); + const body = await response.json() as { error: { code: string } }; + expect(body.error.code).toBe("invalid_excluded"); + }); + + test("rejects unsupported artifact classes instead of querying with arbitrary values", async () => { + const response = await apiGet("/api/lab/artifacts?artifactClass=not-real"); + expect(response.status).toBe(400); + const body = await response.json() as { error: { code: string } }; + expect(body.error.code).toBe("invalid_artifact_class"); + }); +}); diff --git a/tests/mimo-free-provider.test.ts b/tests/mimo-free-provider.test.ts index a97ece2378..cc9477c712 100644 --- a/tests/mimo-free-provider.test.ts +++ b/tests/mimo-free-provider.test.ts @@ -36,10 +36,10 @@ describe("mimo-free provider registry", () => { expect(entry?.defaultModel).toBe("mimo-auto"); }); - test("providerConfigSeed propagates keyOptional and liveModels", () => { + test("providerConfigSeed preserves keyOptional and disables static live discovery", () => { const seed = providerConfigSeed(entry!); expect(seed.keyOptional).toBe(true); - expect(seed.liveModels).toBe(true); + expect(seed.liveModels).toBe(false); }); test("is included in the key-login map", () => { diff --git a/tests/provider-static-model-discovery.test.ts b/tests/provider-static-model-discovery.test.ts new file mode 100644 index 0000000000..93ffee8348 --- /dev/null +++ b/tests/provider-static-model-discovery.test.ts @@ -0,0 +1,118 @@ +import { describe, expect, test } from "bun:test"; +import { createMimoFreeAdapter, MIMO_CHAT_URL } from "../src/adapters/mimo-free"; +import { enrichProviderFromRegistry, providerConfigSeed } from "../src/providers/derive"; +import { getProviderRegistryEntry } from "../src/providers/registry"; +import { + providerMatchesRegistryTransportWithStaticGuards, + providerSupportsLiveModelDiscovery, + registryEntrySupportsLiveModelDiscovery, +} from "../src/providers/static-model-discovery"; +import { routedProviderConfig } from "../src/router"; +import type { OcxProviderConfig } from "../src/types"; + +function provider(overrides: Partial<OcxProviderConfig>): OcxProviderConfig { + return { + adapter: "openai-chat", + baseUrl: "https://example.com/v1", + ...overrides, + }; +} + +describe("static provider model discovery policy", () => { + test("ClinePass and MiMo Free seeds are static without changing the registry catalog", () => { + for (const id of ["cline-pass", "mimo-free"] as const) { + const entry = getProviderRegistryEntry(id); + expect(entry).toBeDefined(); + expect(registryEntrySupportsLiveModelDiscovery(entry!)).toBeFalse(); + expect(providerConfigSeed(entry!).liveModels).toBeFalse(); + } + }); + + test("stale canonical ClinePass discovery is disabled without replacing saved models", () => { + const savedModels = ["cline-pass/kimi-k3", "saved-selector"]; + const config = provider({ + adapter: "openai-chat", + baseUrl: "https://api.cline.bot/api/v1", + authMode: "key", + liveModels: true, + models: [...savedModels], + }); + + enrichProviderFromRegistry("cline-pass", config); + expect(config.liveModels).toBeFalse(); + expect(config.models).toEqual(savedModels); + + const routed = routedProviderConfig("cline-pass", provider({ + adapter: "openai-chat", + baseUrl: "https://api.cline.bot/api/v1", + authMode: "key", + liveModels: true, + models: [...savedModels], + })); + expect(routed.liveModels).toBeFalse(); + expect(routed.models).toEqual(savedModels); + }); + + test("the normal Cline API stays live even though it shares the ClinePass destination", () => { + const config = provider({ + adapter: "openai-chat", + baseUrl: "https://api.cline.bot/api/v1", + authMode: "key", + liveModels: true, + defaultModel: "anthropic/claude-sonnet-4-6", + }); + + expect(providerSupportsLiveModelDiscovery("cline", config)).toBeTrue(); + expect(providerSupportsLiveModelDiscovery("my-cline", config)).toBeTrue(); + }); + + test("legacy canonical MiMo Free auth is repaired without replacing saved models", () => { + const config = provider({ + adapter: "mimo-free", + baseUrl: MIMO_CHAT_URL, + authMode: "local", + liveModels: true, + models: ["mimo-auto", "saved-selector"], + }); + + enrichProviderFromRegistry("mimo-free", config); + expect(config.authMode).toBe("key"); + expect(config.liveModels).toBeFalse(); + expect(config.models).toEqual(["mimo-auto", "saved-selector"]); + + const routed = routedProviderConfig("mimo-free", provider({ + adapter: "mimo-free", + baseUrl: MIMO_CHAT_URL, + authMode: "local", + liveModels: true, + models: ["mimo-auto", "saved-selector"], + })); + expect(routed.authMode).toBe("key"); + expect(routed.liveModels).toBeFalse(); + expect(routed.models).toEqual(["mimo-auto", "saved-selector"]); + }); + + test("same-named custom MiMo Free rows are not claimed by the canonical preset", () => { + const custom = provider({ + adapter: "openai-chat", + baseUrl: "https://example.com/v1", + authMode: "key", + liveModels: true, + models: ["custom-model"], + }); + + expect(providerMatchesRegistryTransportWithStaticGuards("mimo-free", custom)).toBeFalse(); + const routed = routedProviderConfig("mimo-free", custom); + expect(routed.adapter).toBe("openai-chat"); + expect(routed.baseUrl).toBe("https://example.com/v1"); + expect(routed.liveModels).toBeTrue(); + expect(routed.models).toEqual(["custom-model"]); + }); + + test("the bespoke MiMo Free adapter refuses a custom destination", () => { + expect(() => createMimoFreeAdapter(provider({ + adapter: "mimo-free", + baseUrl: "https://example.com/v1", + }))).toThrow("only supports the canonical Xiaomi MiMo Free endpoint"); + }); +}); diff --git a/tests/settings-startup-health-seam.test.ts b/tests/settings-startup-health-seam.test.ts new file mode 100644 index 0000000000..657e49aed7 --- /dev/null +++ b/tests/settings-startup-health-seam.test.ts @@ -0,0 +1,46 @@ +import { expect, test } from "bun:test"; +import { handleManagementAPI, type ManagementApiDeps } from "../src/server/management-api"; +import type { OcxConfig } from "../src/types"; +import { ManagementRequest as Request } from "./helpers/management-auth"; +import { startupHealthFixture } from "./helpers/startup-health"; + +function baseConfig(): OcxConfig { + return { + port: 10100, + defaultProvider: "openai", + providers: { + openai: { + adapter: "openai-chat", + baseUrl: "https://api.example.test/v1", + apiKey: "sk-secret-value", + defaultModel: "gpt-test", + }, + }, + }; +} + +test("settings PUT uses the injected startup-health reader", async () => { + const config = baseConfig(); + let reads = 0; + const expectedHealth = startupHealthFixture({ diagnosticStale: true }); + const deps: ManagementApiDeps = { + saveConfigPreservingClaudeCode: () => {}, + getCachedStartupHealth: async () => { + reads += 1; + return expectedHealth; + }, + }; + const req = new Request("http://127.0.0.1:10100/api/settings", { + method: "PUT", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ streamMode: "eager-relay" }), + }); + + const response = await handleManagementAPI(req, new URL(req.url), config, deps); + + expect(response?.status).toBe(200); + expect(reads).toBe(1); + expect(await response!.json()).toMatchObject({ + startupHealth: { diagnosticStale: true, status: "native" }, + }); +}); diff --git a/tests/settings-stream-mode.test.ts b/tests/settings-stream-mode.test.ts index 1902a1f6cc..e78987183b 100644 --- a/tests/settings-stream-mode.test.ts +++ b/tests/settings-stream-mode.test.ts @@ -29,9 +29,13 @@ import { usageSummaryRetainedStoreSnapshot, } from "../src/server/management/usage-summary-cache"; import { catalogConvergenceFactory } from "./helpers/catalog-convergence"; +import { startupHealthFixture } from "./helpers/startup-health"; let TEST_DIR = ""; const previousHome = process.env.OPENCODEX_HOME; +const readTestStartupHealth: NonNullable<ManagementApiDeps["getCachedStartupHealth"]> = async () => ( + startupHealthFixture() +); function baseConfig(): OcxConfig { return { @@ -58,12 +62,17 @@ function putSettings( headers: { "content-type": "application/json" }, body: JSON.stringify(body), }); - return handleManagementAPI(req, new URL(req.url), config, deps); + return handleManagementAPI(req, new URL(req.url), config, { + getCachedStartupHealth: readTestStartupHealth, + ...deps, + }); } function getSettings(config: OcxConfig): Promise<Response | null> { const req = new Request("http://127.0.0.1:10100/api/settings"); - return handleManagementAPI(req, new URL(req.url), config); + return handleManagementAPI(req, new URL(req.url), config, { + getCachedStartupHealth: readTestStartupHealth, + }); } beforeEach(() => { @@ -84,7 +93,7 @@ afterEach(() => { try { rmSync(TEST_DIR, { recursive: true, force: true }); } catch { - /* Windows may briefly lock while a background startup-health probe exits */ + /* Windows may briefly retain file handles during test cleanup */ } } });