diff --git a/docs/ASVS-L2-PHASE0-CHANGES.md b/docs/ASVS-L2-PHASE0-CHANGES.md index a999a1c3..9aec17f4 100644 --- a/docs/ASVS-L2-PHASE0-CHANGES.md +++ b/docs/ASVS-L2-PHASE0-CHANGES.md @@ -72,9 +72,23 @@ The WS handshake checks `Origin` the same way (§1). ## 4. Key-management & cryptographic inventory (ASVS 11.1.1 / 11.1.2) -A single, change-controlled inventory of every key, algorithm, and certificate the engine relies on. +The engine's change-controlled inventory of the keys, algorithms, and certificates it relies on. Update it whenever a crypto dependency, algorithm, or key source changes. +This section used to open by claiming a single inventory of **every** key, algorithm, and certificate +the engine relies on. That was false as written: four shipped crypto surfaces had no row in the +inventory table below — the verifying SMTP hop shared by the `[alerts]` sink and the EMAIL/DIRECT +destinations, XML-DSig verification via `signxml`, the SFTP connector's SSH host-key trust and its +client key, and the SMART Backend Services client assertion. Rows for all four are in the table now. +Read the table as covering **at least** what it lists rather than as closed, and treat a surface you +cannot find here as uninventoried rather than as absent from the engine. + +**What a deploying site still owns.** These rows describe what the engine generates, loads, and +enforces. Material a site supplies — CA bundles, partner and signer certificates, SSH host keys, PEM +private keys and their passphrases — is generated, stored, escrowed, and destroyed under that site's +own key-management policy; the table records where each input enters the engine and what it is used +for, not how it is protected before it gets there. + > **Enforced by CI (ASVS 11.1.3, WP-L3-02).** `scripts/security/crypto_inventory_check.py` is the > machine-readable companion to this section. It walks the five first-party roots (`messagefoundry/`, > `messagefoundry_webconsole/`, `harness/`, `tee/`, `scripts/` — pinned identical to @@ -104,7 +118,7 @@ Update it whenever a crypto dependency, algorithm, or key source changes. | Engine wheel attestation ([ADR 0041](adr/0041-load-path-attestation-and-change-attribution.md) D3) | SHA-256 over each **loaded** first-party `messagefoundry` module file, compared to the installed wheel's `*.dist-info/RECORD` baseline (a base64 `sha256=` manifest already in the wheel); `hashlib` in `integrity.py` | Drift recorded in the hash-chained `startup_integrity` audit row (not a secret); RECORD baseline read from site-packages metadata | Recomputed at startup + on demand; in-place-tamper tripwire (integrity, not confidentiality). Alert-only by default; `[integrity].fail_closed_on_drift` refuses to start on drift; no-op on an editable install | | ASVS corpus pin ([ADR 0156](adr/0156-asvs-scorecard-as-data-a-derived-count-verified-evidence-anchors-and-a-fail-closed-drift-gate.md)) | SHA-256 over the **OWASP ASVS 5.0.0 corpus file**, recorded in `[scorecard].corpus_sha256` and recomputed on every verifier run; `hashlib` in `scripts/asvs/scorecard.py`. **Integrity of a build input, not a security control** — no secret, no key, no message authentication, and nothing user- or PHI-derived is hashed. It exists because the corpus was originally fetched from `master` (the bleeding-edge branch, where a rolling "latest" release republishes identical filenames) and matched the tagged `v5.0.0_release` asset only by luck; the digest is now recorded and checked rather than assumed, because ASVS requirement ids are **not stable across versions** (bare `1.2.5` is *Architecture* in 4.0.3 and *Encoding and Sanitization* in 5.0.0), so a corpus that moves silently re-points every id in the scorecard | Not a secret: the digest is committed alongside the corpus it pins | Recomputed on every scorecard verification; a mismatch fails the gate and forces re-verification before any verdict is trusted | | ASVS scorecard revision identifier ([ADR 0156](adr/0156-asvs-scorecard-as-data-a-derived-count-verified-evidence-anchors-and-a-fail-closed-drift-gate.md)) | SHA-256 over the **ASVS scorecard file**, printed truncated to 16 hex characters by a `--prove-absences` run; `hashlib` in `scripts/asvs/prove_report.py`. Same class as the corpus pin above and **not a security control** for the same reasons — no secret, no key, no message authentication, nothing user- or PHI-derived. It differs only in what it covers: the record itself rather than a build input, and it is never compared against a declared value. It exists so a run states *which* revision of the record it read — two runs reporting different counts are otherwise indistinguishable from one run whose input moved underneath it | Not a secret: it is an identifier in a run log, and the scorecard it covers is private for unrelated reasons | Recomputed on every run; nothing is gated on it, so a change is information for a reader rather than a failure | -| Outbound message signing (opt-in) | Detached JWS (RFC 7515) — RS256/PS256 (RSA) or ES256 (ECDSA P-256), SHA-256; `cryptography` in `transports/signing.py` (ASVS 4.1.5, [ADR 0018](adr/0018-per-message-signatures-accepted-risk.md)) | Operator-supplied PEM **private** signing key per connection (inline via `env()` or a PEM file path; encrypted-key passphrase via `env()`); the **public** key is shared with the partner out-of-band. **Usage scope:** this private key **only** signs this connection's outbound per-message JWS — a message-**authenticity/integrity** key in transit; it is never used for at-rest encryption or session/token material, and the partner holds only the matching **public** verification half | **OFF by default**; per-connection opt-in. `kid` carried in the JWS header so key rotation / a managed provider ([ADR 0019](adr/0019-pluggable-keyprovider-hsm-kms-vault.md)) slots in without a wire change | +| Outbound message signing (opt-in) | Detached JWS (RFC 7515) — RSA `RS256`/`RS384` (PKCS#1 v1.5, deterministic) or `PS256` (PSS), or ECDSA `ES256` (P-256) / `ES384` (P-384); SHA-256 except `RS384`/`ES384`, which are SHA-384. This row previously listed only the three SHA-256 algorithms, but the shipped `SignatureAlgorithm` enum has five members and `OutboundSigning.algorithm` accepts any of them, so the SHA-384 pair is available for per-message signing too; `cryptography` in `transports/signing.py` (ASVS 4.1.5, [ADR 0018](adr/0018-per-message-signatures-accepted-risk.md)) | Operator-supplied PEM **private** signing key per connection (inline via `env()` or a PEM file path; encrypted-key passphrase via `env()`); the **public** key is shared with the partner out-of-band. **Usage scope:** this private key **only** signs this connection's outbound per-message JWS — a message-**authenticity/integrity** key in transit; it is never used for at-rest encryption or session/token material, and the partner holds only the matching **public** verification half | **OFF by default**; per-connection opt-in. `kid` carried in the JWS header so key rotation / a managed provider ([ADR 0019](adr/0019-pluggable-keyprovider-hsm-kms-vault.md)) slots in without a wire change | | DIRECT S/MIME (opt-in, [ADR 0085](adr/0085-direct-hisp-smime-connector.md)) | CMS **sign-then-encrypt** in `transports/direct.py` (core `cryptography` `serialization.pkcs7`): PKCS#7 signature over the body with a **SHA-256** digest, the public-key signature algorithm (RSA / ECDSA) following the loaded signing key type (not pinned to RSA), then a PKCS#7 **envelope** to the partner's recipient cert. The envelope content-encryption cipher is the **`cryptography` pkcs7 library default** — no algorithm is pinned in code | Sender **signing cert** + PEM **private key** (optional `signing_key_password`) and the per-partner **`recipient_cert`**, all operator-supplied files; the recipient cert is trust-verified at construction against an operator `trust_anchor` (one-level direct-issuance check); key/cert mismatch refused. **Usage scope:** the sender signing key signs the CMS body and the partner's `recipient_cert` encrypts the CMS envelope — this material protects the **confidentiality + authenticity of a DIRECT message to one partner in transit**; it is not an at-rest store key and encrypts nothing in the store | **OFF by default** — only when a DIRECT Connection is configured, and its HISP relay host is gated by the **opt-in** `[egress].allowed_direct` allow-list (empty by default = unrestricted; an unlisted host is refused only once the list is populated, or outright when `[security].block_unlisted_outbound` is set). Signing key + recipient certs rotate on the schedule below | | OIDC IdP JWKS verification keys (opt-in, [ADR 0142](adr/0142-federated-sso-oidc-authorization-code-pkce-relying-party-hybrid-ad-backed.md)) | **Public** verifying keys fetched from the IdP JWKS: **RS256/PS256** (RSA, ≥ 2048-bit floor) and **ES256/ES384** (EC P-256/P-384) — rebuilt from each JWK by `cryptography` in [`auth/oidc/jwks.py`](../messagefoundry/auth/oidc/jwks.py); the closed `SignatureAlgorithm` enum forecloses `alg:none` and RS256→HS256 confusion. Bounded, TTL-cached (`DEFAULT_JWKS_TTL_SECONDS`), a 512 KiB body cap, a global min-refetch floor (fetch-amplification bound), and a hard refusal of a duplicate `kid`; a key below the floor is skipped/refused, never merely warned. **Usage scope:** these are **public**, non-secret keys used **only** to verify the IdP's id-token signature at console login — they encrypt nothing and can protect no data; the engine holds no private half. | Fetched from the IdP JWKS URI over the CA-pinned no-redirect opener (row below); held process-local in `JwksCache`, never persisted, never logged | Refetched per TTL / on an unknown `kid` within the amplification bound; rolls when the IdP rotates its signing keys | | OIDC IdP TLS trust anchor (`[auth].oidc_tls_ca_cert_file`, opt-in, [ADR 0142](adr/0142-federated-sso-oidc-authorization-code-pkce-relying-party-hybrid-ad-backed.md)) | Pins the CA that must anchor the IdP's TLS server cert on **both** federated-SSO legs (JWKS fetch + token endpoint) — the hardened, no-redirect `ssl` opener in [`auth/oidc_http.py`](../messagefoundry/auth/oidc_http.py) (mirrors `ad_tls_ca_cert_file`); unset ⇒ the OS trust store via `truststore`. No insecure/`verify=False` escape exists — the IdP hop carries an authentication assertion. **Usage scope:** a **trust anchor**, not a key the engine holds — it authenticates the IdP endpoint's TLS identity only; it signs and encrypts nothing and protects no at-rest data. | Operator-supplied CA PEM path (`[auth].oidc_tls_ca_cert_file`) or the OS trust store | Managed by the operator / OS trust store; rotate on IdP CA change | @@ -115,15 +129,41 @@ Update it whenever a crypto dependency, algorithm, or key source changes. | Cert tooling — `.pfx` import / read-only inventory / self-signed dev cert (BACKLOG #71/#72) | `cryptography` in [`pki.py`](../messagefoundry/pki.py) — the single PKI call site for the `cert` CLI group: PKCS#12/.pfx import (`pkcs12.load_key_and_certificates`) writes the leaf cert + private key + CA chain to the PEM files the TLS loaders already read; a **read-only** inventory reads only **public** cert facts (subject/issuer/notAfter/SAN/days) via `x509`; `make_self_signed` mints an **EC P-256 / SHA-256** self-signed cert for **non-prod** bring-up. `pipeline/cert_expiry.py` shares this module's `read_cert_facts` (so it no longer imports `cryptography` itself). **Usage scope:** an operator CLI utility — it imports/serializes/inspects operator-supplied cert material and mints throwaway dev certs; it holds no long-lived engine key, signs no message, and encrypts nothing at rest. The imported/minted **private-key** PEM is written `O_EXCL` + `0o600` + the `_secure_file` DACL; the `.pfx` passphrase is env-only (`MEFOR_PFX_PASSWORD`), never a CLI arg, never logged/echoed/put in an exception | Operator-supplied `.pfx` bundle → cert/key/CA PEM files on disk (`--out-dir`) | Managed by the operator / PKI; self-signed dev certs are disposable (default 365-day validity) | | Engine/console seam identity (BACKLOG #1220) | SHA-256 over the **discovered** engine/console contract surface, truncated to 16 hex characters, published as `ENGINE_UI_SEAM` in [`api/_ui_seam.py`](../messagefoundry/api/_ui_seam.py) and derived by `hashlib` in [`scripts/webconsole_seam_snapshot.py`](../scripts/webconsole_seam_snapshot.py). **A change detector, not a security control** — no secret, no key, no message authentication, and nothing user- or PHI-derived is hashed; the input is a serialization of public type signatures, field names, enum members and `Literal` values. It replaced a hand-picked incrementing integer, which two unlanded branches had both claimed for two different contract changes while the golden snapshot auto-merged clean under one value. What it needs is accidental-collision avoidance across the contract surfaces this project will ever produce: at 64 bits the birthday bound is 2.7e-12 for 10,000 distinct surfaces, about 500x the ~20 seam moves to date. Preimage resistance buys nothing — anyone able to craft a colliding surface already has commit access to the file holding the constant. SHA-256 rather than BLAKE2 or a non-approved digest only because the engine renders a `fips_mode` attestation and a non-approved hash in the shipped surface invites a FIPS question for no gain | Not a secret: the digest is committed in source and mirrored in the console's `SUPPORTED_ENGINE_SEAMS` | Recomputed by `scripts/webconsole_seam_snapshot.py --write` whenever the contract changes; a stale value fails `tests/test_webconsole_seam_snapshot.py` | | DAST scan-target credential ([ADR 0155](adr/0155-dast-dynamic-security-testing-of-the-running-engine.md)) | Throwaway per-run password — URL-safe CSPRNG (`secrets.token_urlsafe(24)`, ~32 characters) in [`scripts/security/dast_target.py`](../scripts/security/dast_target.py) — authenticating the two ephemeral identities (`dast-admin`, `dast-viewer`) the authenticated authorization sweep provisions. It is hashed by the same argon2id path as any local account. Generated rather than checked in **because a constant is strictly weaker and goes stale**; it is never logged, never written to the receipt artifact, and cannot reach a real deployment (the sweep creates its own store). Not key material: it protects nothing at rest or in transit and unlocks no key | Generated in-process per run; the optional `MEFOR_DAST_SCAN_PASSWORD` variable is a by-hand-repro escape hatch only. Stored only as the argon2id hash inside the scan's own SQLite store, which the runner creates **empty in a temporary directory** | Lives for exactly one scan — the temporary store is deleted with the run, so there is nothing to rotate | +| Outbound SMTP transport TLS (the `[alerts]` sink, the EMAIL destination, the DIRECT HISP relay) | One verifying `ssl` context per hop, built by `build_smtp_tls_context` in [`config/tls_policy.py`](../messagefoundry/config/tls_policy.py) and passed explicitly to `starttls()`: chain + hostname verification, a TLS 1.2 floor, the approved ECDHE groups, forward-secret cipher suites, and strict RFC 5280 verify flags. All three SMTP call sites route through that one factory ([`pipeline/alert_sinks.py`](../messagefoundry/pipeline/alert_sinks.py), [`transports/email.py`](../messagefoundry/transports/email.py), [`transports/direct.py`](../messagefoundry/transports/direct.py)), so one policy decides every SMTP hop. Passing **no** context is the failure this row exists to foreclose — `smtplib`'s own fallback is `ssl._create_unverified_context` (`CERT_NONE`, `check_hostname=False`), which encrypts the hop without authenticating the relay. **Usage scope:** a **trust anchor**, not a key the engine holds — it authenticates the relay's TLS identity on the alert / security-notification / DIRECT-relay hops only; it signs nothing, encrypts nothing at rest, and has no private half. | The connection's own CA PEM (`[alerts].email_tls_ca_file`, a connector `ca_file`), else the instance `[tls].internal_ca_file`, else the OS trust store | Managed by the operator / OS trust store; rotate on relay CA change. Turning verification off is a **loosening**, and the callers gate it differently: the `[alerts]` sink needs `[security].allow_unverified_alert_smtp_tls` at the serve gate, while the EMAIL/DIRECT connectors read the clamped `MEFOR_ALLOW_INSECURE_TLS` escape | +| Inbound XML-DSig verification (opt-in, the `[xml]` extra) | Enveloped XML-DSig signature + digest verification through `signxml` (which delegates to `cryptography`) in [`parsing/xml/signature.py`](../messagefoundry/parsing/xml/signature.py) — `signxml.XMLVerifier().verify()`, called on demand by a Handler against a signed XML/SOAP body that has already been through the hardened lxml parser (XXE/DTD lockdown). **A trust anchor is required and its absence is refused:** `verify()` raises `ValueError` unless the caller pins the expected signer certificate (`x509_cert`) or names a partner CA (`ca_pem_file`), because signxml's own default would accept any signature chaining to the host's system CA store. A failed verification is returned as data (`XmlSignatureResult`) with a PHI-safe reason category, never the document. **Usage scope:** verification material only — the engine holds **no** private half on this path, signs nothing, and encrypts nothing; it establishes the origin and integrity of an inbound document and nothing else. | Operator-supplied signer certificate or partner CA PEM, passed per call by the Handler; the engine stores neither | Rotate at the partner. `signxml` ships behind the optional `[xml]` extra, so a base install has no XML-DSig path at all. **Operator-owned:** choosing the anchor is the deploying site's decision — a Handler that passes a broadly-trusted CA verifies far more signers than it intends, and nothing in the engine can detect that | +| SFTP transport — SSH host-key trust and client key (opt-in, the `[sftp]` extra) | `paramiko` in [`transports/remotefile.py`](../messagefoundry/transports/remotefile.py) (`_SftpClient`). **Server authentication:** the system `known_hosts` plus an optional per-connection `known_hosts` file under paramiko's `RejectPolicy`, so an unknown host key is refused at connect. `AutoAddPolicy` is reachable only through the `MEFOR_ALLOW_INSECURE_TLS` escape read via the [ADR 0092](adr/0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md) clamp — inert wherever the construction posture is an enforcing PHI hop — and it logs a warning where it does apply. **Client authentication:** an operator-supplied **RSA** private key (`private_key`, passphrase `key_password`) loaded via `paramiko.RSAKey.from_private_key`, or a password; RSA is the only key type the loader builds today, so an Ed25519 or ECDSA key is not usable here. `allow_agent=False` and `look_for_keys=False`, so no ambient agent key or `~/.ssh` key is ever picked up. **Usage scope:** transport authentication for one SFTP endpoint — the client key proves *this engine* to that server; it encrypts nothing at rest and signs no message. | Host keys from the OS `known_hosts` and the connection's `known_hosts` file; the client key is `env()`-sourced PEM (`private_key`) with `key_password`, both `/metadata`-redacted | Rotate at the partner; re-pin the host key on a server rebuild. `paramiko` behind the optional `[sftp]` extra. **Operator-owned:** the engine can refuse an unknown host key but cannot tell a correct pinned key from a wrong one — populating `known_hosts` is the deploying site's job | +| SMART Backend Services client assertion (opt-in, [ADR 0024](adr/0024-smart-backend-services-token-provider.md)) | A signed compact JWT minted by `CompactJwtSigner` in [`transports/signing.py`](../messagefoundry/transports/signing.py) and exchanged for a short-lived bearer at the authorization server's token endpoint ([`transports/smart.py`](../messagefoundry/transports/smart.py)). `smart_algorithm` defaults to **RS384**; `ES384` is the other algorithm SMART Backend Services requires, and the setting accepts any of the five JWS algorithms the ADR 0018 signer supports, so pinning a SHA-256 member is possible and is a deliberate departure from the SMART profile. Signing runs over core `cryptography`, no new dependency. The assertion lives 240 seconds (under SMART's 5-minute ceiling), its `aud` is bound to the operator-pinned `token_url` so it is not replayable at another authorization server, that endpoint rides the same `[egress].allowed_http` allow-list as the FHIR data host, and a cleartext `http` token endpoint is refused by the instance security posture unless the hop is attested or cleartext is explicitly accepted with a reason. **Usage scope:** this private key **only** signs this connection's client assertion — it authenticates *the engine* to one FHIR authorization server. It is not an at-rest, session, or message-signing key, decrypts nothing, and the FHIR server holds only the matching public half. | Operator-supplied PEM private key `smart_private_key` via `env()` (passphrase `smart_private_key_password`), `/metadata`-redacted; the minted assertion and the access token are never logged or persisted | **OFF by default** — present only when a SMART-authorized FHIR outbound is configured. Cadence in the rotation schedule below: register the new public key at the FHIR server first, then replace the PEM | | Engine-shard lane ownership ([ADR 0073](adr/0073-ownership-scoped-recovery-single-consumer-lanes.md)) | Rendezvous (HRW) hash — SHA-256 (`hashlib` in `pipeline/sharding.py`) over `destination + shard id`, picking each outbound lane's single delivering shard. A stable, process-independent hash is required (the salted builtin `hash()` would let two shards disagree on an owner); deterministic placement, **not** a security control | No key material — pure function of config names | Recomputed per process from the loaded config; changes only with the shard universe (coordinated fleet restart) | ### Store-key management policy (NIST SP 800-57 alignment) - **Generation** — mint with `messagefoundry gen-key` (32 bytes from `os.urandom`); supply via `MEFOR_STORE_ENCRYPTION_KEY`, **never** the TOML file. -- **Storage / access** — environment only; the process account is the trust boundary. Restrict the - data-volume and service account (see [SERVICE.md](SERVICE.md)); volume encryption - (BitLocker/LUKS) is the required at-rest layer for the columns outside the cipher. +- **Storage / access** — the process account is the trust boundary. This bullet used to say + "environment only"; that is **no longer true**, and an operator who read it would miss shipped + options. There are **at least four** routes today: the `MEFOR_STORE_ENCRYPTION_KEY` environment + value (the cross-platform default); a Windows DPAPI-protected file named by + `[store].encryption_key_file` and written by `messagefoundry protect-key` — a *path*, not a secret, + so it may live in the TOML file, and the environment value wins when both are set; + `[store].key_provider = vault`, which envelope-decrypts a **wrapped** DEK against a non-extractable + key inside HashiCorp Vault/OpenBao Transit (behind the optional `[vault]` extra, so a base install + pulls no SDK); or `[store].cipher_provider = vault_transit`, where there is no local key material at + all, because the bulk encrypt/decrypt happens inside Transit and `encryption_key` / `key_provider` go + unused. `[store].key_provider` also accepts `auto` (the default: environment then DPAPI) and the + pinned `env` / `dpapi`. The remaining external provider names — `aws_kms`, `azure_kv`, `gcp_kms`, + `pkcs11` — are designed but **not built**, and selecting one fails closed at store open rather than + degrading to the plaintext cipher. Restrict the data-volume and service account (see + [SERVICE.md](SERVICE.md)); volume encryption (BitLocker/LUKS) is the required at-rest layer for the + columns outside the cipher. +- **Distribution / holders** — the engine needs exactly **one** holder: the process account, by + whichever route above the site chose. Where the engine reports on the key it reports a fingerprint + and not the key — the security-posture view carries only the active key's one-way `key_id` + (`CipherInfo` in [`store/crypto.py`](../messagefoundry/store/crypto.py)). The escrow requirement in + the next bullet is therefore a **deliberate second holder**, and the only copy outside the running + process this project asks for: escrow once, into the narrowest custody the site's recovery procedure + needs. Every further copy — a value pasted into a ticket, a backup of the config directory, a secret + manager replicated across regions — is a holder the engine can neither see nor revoke, and counting + them is the deploying site's responsibility, not something the engine can do for it. - **Rotation / retirement** — **built (WP-5, ASVS 11.2.2).** Each ciphertext is self-identifying via its `key_id` (SHA-256 fingerprint), so multiple keys coexist: set the new key as `MEFOR_STORE_ENCRYPTION_KEY`, move the prior one to `MEFOR_STORE_ENCRYPTION_KEYS_RETIRED` @@ -137,6 +177,53 @@ Update it whenever a crypto dependency, algorithm, or key source changes. connection labels) — never a body — which relies on volume encryption (accepted residual — see [PHI.md §3](PHI.md#3-encryption-at-rest)). +### Key management for the other private keys the engine loads + +The bullets above govern exactly **one** key — the store DEK — and they do not generalize. The +schedule below has always carried the other keys, but it answers a single question about them (how +often to rotate, and what to replace) and is silent on the rest: who mints them, where they may live, +how many holders are acceptable, and what retiring one actually means. That is this subsection. + +**What the engine mints.** Of the long-lived asymmetric and at-rest key material on this page, the +engine mints exactly two things; every other key below is minted by the deploying site or by a +counterparty, and the engine only loads it. (Short-lived CSPRNG secrets the engine also generates — +session tokens, WebAuthn ceremony challenges, the throwaway scan credential — have their own rows in +the inventory table above.) The two are the store DEK (`messagefoundry gen-key` — 32 bytes from +`os.urandom`, base64-encoded, printed to stdout and not persisted by the command; `generate_key` in +[`store/crypto.py`](../messagefoundry/store/crypto.py)), and a **non-production** self-signed EC P-256 +certificate and key (`messagefoundry cert self-signed`, `make_self_signed` in +[`pki.py`](../messagefoundry/pki.py) — the key PEM is unencrypted PKCS#8, written `O_EXCL` with mode +`0o600` and a tightened Windows DACL, refusing to overwrite an existing file; a self-signed +certificate has no chain of trust and must never front production PHI). `messagefoundry cert import` +unpacks an operator's PKCS#12 bundle into the PEM files the TLS loaders read — it relocates key +material, it creates none. + +| Key | How it reaches the engine | Who holds it | What losing it costs | +|---|---|---|---| +| **Per-message JWS signing key** — the `with_signing` parameters `private_key` / `private_key_password`, which reach a REST/SOAP outbound as the flat settings `sign_private_key` / `sign_private_key_password` ([ADR 0018](adr/0018-per-message-signatures-accepted-risk.md)) | inline PEM through `env()`, **or** a path to a PEM file the OS protects; read once when the connector is constructed (`_load_private_key` in [`transports/signing.py`](../messagefoundry/transports/signing.py)) | the engine process, plus wherever the operator keeps the PEM. Only the public-verifiable signature leaves the box; the receiver verifies against the matching **public** key, exchanged out of band per partner contract | re-issuing the key and republishing the public half to each partner. No stored message becomes unreadable | +| **SMART Backend Services client-assertion key** — `smart_private_key` (+ `smart_private_key_password`) ([ADR 0024](adr/0024-smart-backend-services-token-provider.md)) | the same two forms, resolved through `env()` and loaded by the same signer | the engine process and the operator's secret store; only the signature and the registered `kid` leave the box | re-registering the replacement key with the FHIR server | +| **DIRECT S/MIME sender signing key** — `signing_key` (+ `signing_key_password`) ([ADR 0085](adr/0085-direct-hisp-smime-connector.md)) | a **file path only** — the engine reads the PEM/DER bytes from disk at construction and checks the key against `signing_cert`, refusing a pair whose public keys differ. `signing_key` names a path, so it is deliberately **not** a secret setting: it is the file, not the setting, that has to be protected | the operator's file system, read by the process account; the recipient holds the sender's certificate | replacing the key and certificate and re-exchanging with partners. This connector **sends** only, so nothing already received depends on it | +| **SFTP client key** — `private_key` (+ `key_password`) on a remote-file connection | **inline PEM only** — the value is parsed as key text and never opened as a path — and only an **RSA** key loads today, so an Ed25519 or ECDSA client key is not usable with this connector | the operator's secret store and the engine process; the peer holds the matching public key | enrolling a new public key with the SFTP peer | +| **API TLS server key** — `[api].tls_key_file` (+ `MEFOR_API_TLS_KEY_PASSWORD`) | a PEM file path handed to the TLS stack when the listener's context is built; the key may instead be embedded in the certificate PEM, in which case `tls_key_file` is omitted. A bad PEM, or a wrong or missing passphrase, raises at that construction point — **before** the socket opens — rather than degrading to plaintext | the operator's file system, read by the process account | re-issuing from the site's CA; the listener will not start until a usable key and certificate are present | + +**Distribution.** For all five the engine needs **one** holder: the process account, reading an +`env()`-resolved value or a file. It escrows none of them, and none should be escrowed on the DEK's +reasoning — the DEK is escrowed because losing it strands rows nothing else can recover, whereas +losing any of these five costs a re-issue and a conversation with a counterparty, never data. The +right holder count for them is the lowest the site can operate with. The connection settings that +carry key text or a passphrase are `/metadata`-redacted (`_SECRET_SETTING_KEYS` and +`_is_secret_setting` in [`config/wiring.py`](../messagefoundry/config/wiring.py)), so a console +operator cannot read a key value back out of a running engine; the **path**-valued connection setting +`signing_key` is served as the path it is, which is exactly why the file behind it needs file-system +permissions of its own. + +**Destruction and retirement belong to the deploying site.** The engine ships no command that +destroys, revokes or expires any of these five. It stops using one when the connection's setting +stops naming it, and the PEM keeps working anywhere it was copied — so retiring a key means revoking +or de-registering it at the counterparty and deleting the operator-side copies. Only the store DEK +has the property the Destruction bullet above describes, where discarding the key is itself the +erasure. + ### Rotation schedule (ASVS 13.1.4 / 13.3.4) A rotation cadence per critical secret, justified against the threat model + HIPAA. These are diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 302419a7..fbde2455 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -562,7 +562,7 @@ document ([SECURITY-DOCS-POLICY.md](SECURITY-DOCS-POLICY.md)). | `password_min_length` | int | 15 | local-password policy — ASVS 5.0-aligned, length-first | | `password_require_uppercase` / `password_require_lowercase` / `password_require_digit` / `password_require_symbol` | bool | `false` | character classes — **opt-in**, each independently (ASVS 5.0 forbids mandatory composition); turn one on only for a legacy standard that still mandates it | | `password_check_breached` | bool | `true` | reject known common/breached passwords against a bundled offline top-10k list (no live HIBP call) | -| `password_check_context` | bool | `true` | reject passwords containing app/vendor/HL7 terms (e.g. `messagefoundry`, `mefor`, `hl7`, `corepoint`) | +| `password_check_context` | bool | `true` | reject a local password that **contains** any deny-list term — a case-insensitive substring test, anywhere in the value, not a whole-word or prefix match. The **twelve** terms are listed in full in [SECURITY.md](SECURITY.md) "Password policy"; an earlier revision of this row called them "app/vendor/HL7 terms" and gave four examples, which mis-stated the rule (five of the twelve are generic credential words unrelated to this application or to HL7). The list is fixed in code (`CONTEXT_WORDS` in [`auth/policy.py`](../messagefoundry/auth/policy.py)): **this flag turns the whole check on or off, and no setting adds or removes a term**, so a site needing its own vocabulary uses `password_breach_corpus_file` below — which matches the *whole* password, never a substring | | `password_check_username` | bool | `true` | reject a password containing the user's **own username** (ASVS 6.2.11) | | `password_breach_corpus_file` | path | — | optional path to a **larger offline breach corpus** that augments the bundled top-10k list (ASVS 6.2.12): a plaintext list **or** an HIBP-style SHA-1 hash export (`HASH[:count]` lines, auto-detected). Fully offline — still no live HIBP call. Use a curated subset, not the full ~40 GB HIBP set (it is loaded into memory). A path, not a secret | | `lockout_threshold` | int | 5 | failed logins before lock (per account) | @@ -716,11 +716,19 @@ disposition, and the audit trail stay intact — the Mirth Data-Pruner pattern); `messages` row and never touches a body still in flight. The *row* survives; its PHI *columns* do not — `messages.metadata` is nulled in the same statement as the body (ASVS 14.2.7). The raw `[retention]` fields still default to `0`/`""` = keep/off, **but `serve` applies a posture gate on top of them, so retention is *not* -opt-in on a PHI instance**: under `[security].enforcement = enforce` (the default) an unbounded -`[security].delete_message_bodies_after_days` or `[retention].dead_letter_days` **refuses to start -(exit 2)**; on a non-enforcing PHI instance each *unset* window is auto-bounded to **30 days**. All -three built-in environment names (`dev`, `staging`, `prod`) derive PHI. The audited opt-out is -`[security].allow_keeping_phi_indefinitely = true`. See [PHI.md §8](PHI.md#8-retention--purge). +opt-in on a PHI instance**: each *unset* window that carries an auto-bound — +`[security].delete_message_bodies_after_days`, `[retention].dead_letter_days` and +`[retention].reference_snapshot_days` — is **defaulted to 30 days** at startup, under **both** +`[security].enforcement` dials, and the defaulted settings are named on stderr. A window set +**explicitly to `0`** is not defaulted: that **refuses to start (exit 2)** under `enforce`, and warns +under `warn`. This paragraph used to state the opposite split — refusal under `enforce`, auto-bound +only on a non-enforcing instance — which the shipped gate in +[`__main__.py`](../messagefoundry/__main__.py) refutes; an *unset* window has not refused since the +auto-bound moved to both dials. All three built-in environment names (`dev`, `staging`, `prod`) +derive PHI. The audited opt-out is `[security].allow_keeping_phi_indefinitely = true`, which +suppresses the auto-bound as well as the refusal. **Thirty days is the engine's floor against an +accidentally unbounded window, not your retention policy — set each window to the number your site +actually requires.** See [PHI.md §8](PHI.md#8-retention--purge). | Key | Type | Default | Notes | |---|---|---|---| | `messages_days` | | | **→ moved to `[security].delete_message_bodies_after_days`** (ADR 0118) — set it there; no longer accepted in `[retention]`. | @@ -1518,7 +1526,7 @@ and a PHI weakening under **strict enforcement** (`enforcement = enforce`, the d | `sign_out_after_idle_minutes` | int | `30` | session idle timeout | | `max_session_hours` | int | `12` | session absolute lifetime | | `block_unlisted_outbound` | bool | `true` | deny-by-default egress — only allow-listed destinations send. **Leaving it unset does not apply `true`** — the internal flag stays `false` and the `[egress]` startup gate decides; see the note under this table | -| `delete_message_bodies_after_days` | int | `30` | bounded PHI-body retention; `0` = keep indefinitely (audited). **Leaving it unset does not apply 30** — the internal window stays `0` and the `[retention]` startup gate decides (refuse under `enforce`, auto-bound to 30 under `warn`); see the note under this table | +| `delete_message_bodies_after_days` | int | `30` | bounded PHI-body retention; `0` = keep indefinitely (audited). **Leaving it unset does not apply 30 through the desugar** — the internal window stays `0`, and the `[retention]` startup gate then defaults it to 30 days on a PHI instance under **either** enforcement dial. This row used to say the gate refuses under `enforce` and auto-bounds only under `warn`; it does not — only an **explicit** `0` reaches the refusal. See the note under this table | | `allow_keeping_phi_indefinitely` | bool | `false` | audited escape: unbounded PHI retention | | `audit_all_authorization_decisions` | bool | `false` | ePHI access is **always** audited regardless of this switch; this adds full *authorization-decision* tracing on top (off by default — forcing it on risks flooding the audit log). "Always audited" is about **coverage**, not about how hard those rows are to alter afterwards: the audit chain is only cryptographically tamper-*evident* on a **keyed** store, and its verify does not catch a truncated tail — see [`[integrity]`](#integrity) | | `handles_real_patient_data` | bool | *derived* | the master data-class lever (was `[ai].data_class = "phi"`). Unset ⇒ derived from the environment name — **all three built-in names (`dev`/`staging`/`prod`) now derive PHI** ([ADR 0148](adr/0148-phi-default-posture-and-an-explicit-security-enforcement-level.md) GIVEN 1, so the default/CI path exercises the encryption/egress/retention controls rather than first meeting them in production); a genuinely-synthetic dev/CI box must set `false` **explicitly** (a loud, audited opt-out), and a custom-named env must declare it | @@ -1534,7 +1542,7 @@ and a PHI weakening under **strict enforcement** (`enforcement = enforce`, the d > | Row | Reads as | Internal field with `[security]` absent | What an unconfigured PHI instance actually does | > |---|---|---|---| > | `block_unlisted_outbound` | `true` | `egress.deny_by_default = False` | the [`[egress]`](#egress) gate decides: with none of the six **counted** `allowed_*` lists `serve` **exits 2** (`allowed_smtp`/`allowed_direct` do not count); with ≥1 counted list it **flips deny-by-default on** for the transports you left empty | -> | `delete_message_bodies_after_days` | `30` | `retention.messages_days = 0` | under `enforcement = enforce` (the default) an unbounded window **refuses to start (exit 2)**; under `warn` each *unset* window is auto-bounded to 30 days; a synthetic instance keeps bodies forever | +> | `delete_message_bodies_after_days` | `30` | `retention.messages_days = 0` | the [`[retention]`](#retention) gate defaults each *unset* window to 30 days on a PHI instance under **both** enforcement dials; an **explicit** `0` refuses to start (exit 2) under `enforce` and warns under `warn`; a synthetic instance keeps bodies forever. This cell previously had the refuse / auto-bound split backwards | > > Neither is a silent fail-open — both paths end in a refusal or an audited flip, and `serve` > back-fills the `[security]` object from the resolved internal values before serving, so @@ -1604,13 +1612,19 @@ A **complete, startable** `messagefoundry.toml` for a loopback PHI instance on a run as `messagefoundry serve --config --env prod` (the active environment is required and has no default; `--env` is the CLI layer over `[ai].environment`, which is why it is not in the file). -**Three of these blocks exist only because a shipped serve gate refuses without them** — they are not -optional garnish. An earlier version of this example carried none of the three and would have hit -`exit 2` three times over. The four gates a stock PHI instance meets, and what satisfies each: +**Two of these blocks exist only because a shipped serve gate refuses without them** — they are not +optional garnish. An earlier version of this example carried neither and would have hit `exit 2` +twice over. `[retention]` is here for a different reason, given on its own line below. This paragraph +previously said three blocks and four refusing gates; the retention gate stopped refusing over an +unset window when the 30-day auto-bound moved to both enforcement dials. The gates a stock PHI +instance meets, and what satisfies each: **keyless PHI** → `MEFOR_STORE_ENCRYPTION_KEY` in the environment; **open egress** → at least one *counted* `[egress]` list (see the [`[egress]`](#egress) ⚠️ — `allowed_smtp` alone does not count); -**unbounded retention** → `[security].delete_message_bodies_after_days` **and** -`[retention].dead_letter_days`; **no security-notification channel** → the `[alerts]` SMTP transport. +**unbounded retention** → nothing you must configure to boot: `serve` defaults each *unset* PHI +window to 30 days rather than refusing, and only an **explicit** `0` is refused (see +[`[retention]`](#retention)) — set `[security].delete_message_bodies_after_days` and +`[retention].dead_letter_days` anyway, so the windows carry your site's numbers instead of the +engine's floor; **no security-notification channel** → the `[alerts]` SMTP transport. `[logging]` and `[api]` here are illustrative, not gate-required. `backend = "sqlserver"` also needs the `sqlserver` extra + ODBC Driver 18 installed (see the note under [`[store]`](#store--message-store--db)). @@ -1662,7 +1676,8 @@ forward_tls_ca_file = "C:/mefor/siem-ca.pem" # required for tls unless forward [retention] # The inbound-body window is [security].delete_message_bodies_after_days above — setting # messages_days here is REJECTED at load (ADR 0118). Only the plumbing keys stay in this section: -dead_letter_days = 90 # REQUIRED: an unbounded dead-letter window is its own exit 2 under enforce +dead_letter_days = 90 # RECOMMENDED: left unset this is auto-bounded to 30 days; an explicit 0 is + # an exit 2 under enforce. Set it so the window is your number, not the engine's # NOTE: vacuum_at / wal_checkpoint_seconds are SQLite-only and a documented NO-OP on this # backend = "sqlserver" store — space reclamation is a DBA operation there. Deliberately not set. ``` diff --git a/docs/CONNECTIONS.md b/docs/CONNECTIONS.md index 513d360d..8043d4a1 100644 --- a/docs/CONNECTIONS.md +++ b/docs/CONNECTIONS.md @@ -715,14 +715,23 @@ at the source. #### File handling & quarantine policy (ASVS 5.1.1) -MessageFoundry's file surface has three parts: the **directory sources** (the local `File(...)` and -remote `Sftp(...)`/`Ftp(...)` connectors) that ingest drop-directory files into the pipeline; the +MessageFoundry's file surface has **at least four** parts. This list is maintained by hand, so read it +as the current inventory and not as a closed set. The **directory sources** (the local `File(...)` and +remote `Sftp(...)`/`Ftp(...)` connectors) ingest drop-directory files into the pipeline; the **opt-in HTTP uploaded-logs upload** (POST `/uploads` + the web-console delegate POST `/ui/uploaded-logs/upload`, [ADR 0134](adr/0134-offline-uploaded-logs-viewer-connection-decoupled-upload-browse-resend-deletion-phi-at-rest-posture-stdlib-multipart.md)) -for operator diagnostic logs; and the **attachment download** route (GET +carries operator diagnostic logs; the **attachment download** route (GET `/messages/{message_id}/attachments/{attachment_id}`, [ADR 0105](adr/0105-streaming-very-large-hl7-attachments-detach-the-opaque-document-from-the-transformable-skeleton.md)) -that serves a detached document back out. The **directory source's** handling of an untrusted drop -directory is fixed policy (the HTTP uploaded-logs surface has its own policy block below): +serves a detached document back out; and the **DICOM C-STORE SCP** (an inbound `DICOM(...)`, +[ADR 0025](adr/0025-dicom-codec-store-connectors.md)) receives whole objects **pushed by a remote +modality** over DIMSE. An earlier revision of this sentence said "three parts" and omitted the SCP; +that enumeration was wrong — the SCP is a receiver of remote-pushed content on the same footing as the +two HTTP routes. None of the drop-directory policy below applies to it: its size ceilings, peer +controls and transport security are connector settings documented under +[DICOM](#dicom--dicom-inbound-c-store-scp--outbound-c-store-scuc-echo-and-dicomweb-stow-rs-adr-0025), +and a deploying site must set them there rather than assume this block covers them. The **directory +source's** handling of an untrusted drop directory is fixed policy (the HTTP uploaded-logs surface has +its own policy block below): - **Permitted type — the inbound's declared `content_type` (default `hl7v2`).** Files are selected by the `pattern` glob (default `*.hl7`), and every candidate is **content-sniffed against that declared @@ -739,7 +748,17 @@ directory is fixed policy (the HTTP uploaded-logs surface has its own policy blo - **Maximum size.** `max_file_bytes` (default **16 MiB**, matching the MLLP frame cap). An oversize file is rejected by a `stat()` **before** it is read into memory (OOM / DoS guard); `None`/`0` disables it. - **Decompression is off by default; opt-in single-stream gzip is bomb-guarded** (ADR 0123). With no - `decompress=` set the connector reads raw bytes only and there is no unpacked-size surface. When + `decompress=` set the connector performs no decompression itself, so it materialises nothing beyond + `max_file_bytes` where that cap is set. An earlier revision went further and said there is "no + unpacked-size surface"; that was wrong — a file's *payload* can carry its own compressed stream. The + shipped case is a **Deflated Explicit VR LE** DICOM object, which the drop's content sniff accepts on + the `DICM` magic alone and which therefore reaches the pipeline with its inflated size unexamined. + That inflate is bounded where the object is unpacked rather than at ingest: at **16 MiB**, with no + per-connection knob, when a Router or Handler parses it (`guard_part10_deflate` in + `parsing/dicom/_inflate.py`, called from `DicomPeek.parse` and `DicomDataset.parse`), and at + `max_object_bytes` when an outbound C-STORE SCU forwards it. A site dropping DICOM into a watch + directory should size those two ceilings deliberately rather than read this bullet as saying no + unpacking happens. When `decompress="gzip"` is enabled it gunzips each drop **before** the content sniff, the AV scan, and the batch split (so all three see the real bytes), and `max_decompressed_bytes` (default 64 MiB) caps the *decompressed* size — a decompression-bomb guard the compressed-only `max_file_bytes` cap cannot @@ -1602,7 +1621,7 @@ MWL, Query/Retrieve (C-FIND/C-MOVE/C-GET), and pixel-data handling. | `presentation_contexts` | `None` → SR + common image storage + Verification | the SOP classes the SCP negotiates (transfer syntaxes default to the standard set) | | `calling_ae_allowlist` | `None` → any (subject to the IP gate) | only these calling AE titles may associate (fail-closed when set) | | `require_called_ae_title` | `True` | a peer must address this engine's `ae_title` as the called AE | -| `max_object_bytes` | `134217728` (128 MiB) | reject a single C-STORE object larger than this **before** the durable commit (OOM/DoS guard) | +| `max_object_bytes` | `134217728` (128 MiB) | reject a single C-STORE object larger than this **before** the durable commit (OOM/DoS guard). It is **also** the ceiling for the pre-decode **inflate** of a *Deflated Explicit VR LE* object: the SCP bound-inflates the raw received Data Set before pydicom touches it, so an over-cap deflate bomb is a DIMSE failure and is never decoded or committed. Note that `0`/`None` does not simply widen this — it removes the object-size check entirely **and tightens** the inflate ceiling to the codec default of **16 MiB**, which is what the guard falls back to when no object cap is configured | | `max_associations` | `10` | cap on concurrent inbound associations (connection-flood guard) | | `max_pdu_size` | `16384` | cap one PDU's bytes (`0` = unbounded); DoS guard | | `timeout_seconds` | `30.0` | ACSE/DIMSE/network timeout | diff --git a/docs/PHI.md b/docs/PHI.md index 9e7fd2e9..42166451 100644 --- a/docs/PHI.md +++ b/docs/PHI.md @@ -1052,12 +1052,31 @@ anywhere**. Config: [CONFIGURATION.md](CONFIGURATION.md#retention). **"Off by default" is no longer the whole truth on a PHI instance.** The raw `[retention]` fields do still default to `0`, but `serve` applies a posture gate on top of them: -- On a **non-enforcing PHI instance**, each **unset** PHI-body window is **auto-bounded to 30 days** - (secure-by-default) — an explicitly-set value, including an explicit `0`, is respected and only warns. -- On a **PHI instance under `[security].enforcement = enforce`** (the default) with either PHI-body - window unbounded, `serve` **refuses to start (exit code 2)**. +- On **any** PHI instance, under **both** `[security].enforcement` dials, each **unset** window that + carries an auto-bound is **defaulted to 30 days** at startup, and the defaulted settings are named + on stderr. The three are `[security].delete_message_bodies_after_days`, + `[retention].dead_letter_days` and `[retention].reference_snapshot_days`, generated from + [config/retention_classification.py](../messagefoundry/config/retention_classification.py) + (`auto_bounded_windows`) rather than listed at the gate. +- These bullets used to say the auto-bound applied only to a *non-enforcing* PHI instance, and that a + PHI instance under `enforcement = enforce` with a PHI-body window unbounded **refused to start (exit + code 2)**. Both halves were wrong about the shipped code: the auto-bound in + [`__main__.py`](../messagefoundry/__main__.py) is keyed on the retention opt-out alone + (`if not settings.retention.allow_unbounded_phi:`), not on the enforcement dial. A site that read + the old text and deliberately left a window unset — expecting the refusal to hold the boot until + someone chose a number — would instead get a started instance that begins purging PHI bodies at 30 + days. +- What survives is the fail-closed path for an **explicit** `0`. An explicitly-zeroed window is not + auto-bounded, and `serve` then **refuses to start (exit code 2)** under `enforcement = enforce`, or + warns and continues under `warn`. So "unbounded by accident" is still prevented; "unbounded by + inattention" becomes "30 days by inattention". +- The classified windows that carry **no** auto-bound are warned about at startup and left alone — + never silently defaulted and never refused over. `[retention].state_max_age_days` and + `[retention].search_preset_days` are the clearest case, because each keys on a timestamp that only + moves on a **write**, so a silent default would delete data a Handler is still reading; the others + are excluded for their own per-window reasons recorded alongside the classification. - The explicit, **audited** opt-out is `[security].allow_keeping_phi_indefinitely = true`, which - downgrades the refusal to a loud audited warning and suppresses the auto-bound. + suppresses the auto-bound **and** downgrades the refusal to a loud audited warning. - The canonical operator-facing home of the message-body window is now **`[security].delete_message_bodies_after_days`**; its *model* default is 30, but the desugar is **presence-gated** — only an EXPLICITLY-set switch is written through — so an **unset** @@ -1065,7 +1084,13 @@ still default to `0`, but `serve` applies a posture gate on top of them: refusal) is what actually bounds a PHI instance. An explicitly-set value writes through onto `[retention].messages_days`. `[retention].dead_letter_days` stays at its own home. -So a PHI instance cannot run with PHI-body retention "off" without a loud, audited opt-out. +So a PHI instance cannot run with PHI-body retention "off" **by accident** — an unset window is +bounded for you. It can still be run that way deliberately: an explicit `0` plus the audited opt-out +under `enforce`, or an explicit `0` alone under `warn`, which warns and starts. + +**Thirty days is the engine's floor against an accidentally unbounded window, not a retention +policy.** The engine picks that number; a deploying site with a retention obligation of its own must +set each window explicitly, because an auto-bounded window is a number nobody decided. **The pass itself.** It is **leader-gated twice** — at entry and again immediately before the purges, so a node demoted mid-pass never nulls PHI as a stale ex-leader. An optional between-phase wall-clock cap @@ -1302,7 +1327,7 @@ separate follow-up. | Item | Closes | Maps to | Effort | |---|---|---|---| | **P2-1** TLS on the engine API | Tokens + PHI cleartext over the network | §164.312(e) · SC-8 | M | -| **P2-2** MFA for console/API auth — ✅ **Built (WP-14, native TOTP, local accounts)** | Single-factor auth (mitigated for local accounts: `[auth].require_mfa` gates **step-up / sensitive admin operations** for the Administrator role — not every PHI read; AD MFA delegated) | §164.312(d) · IA-2(1) (NPRM-mandated) | M–L | +| **P2-2** MFA for console/API auth — ✅ **Built (WP-14, native TOTP, local accounts)** | Single-factor auth (mitigated for local accounts: `[security].require_mfa` is an **access gate on every authorized route**, and its shipped `require_mfa_scope` is `every_local_account`, not the Administrator role alone; AD MFA delegated) | §164.312(d) · IA-2(1) (NPRM-mandated) | M–L | | **P2-3** Network-segmentation guidance + periodic integrity checks | Lateral movement; tamper detection | §164.312(c) · SC-7/SI-7 | S–M | | **P2-4** Strict-parse CPU/time budget on the hl7apy path | Malformed input pinning a worker — message size/segment caps are built, but the opt-in strict parse itself has no time bound | NIST SC-5 (DoS; not a §164.312 safeguard) | S | diff --git a/docs/SECURITY-LOOSENING.md b/docs/SECURITY-LOOSENING.md index 4515dc7f..03f0fcc7 100644 --- a/docs/SECURITY-LOOSENING.md +++ b/docs/SECURITY-LOOSENING.md @@ -216,10 +216,12 @@ trail. - **When acceptable:** a documented retention requirement that genuinely needs keep-forever, accepted in writing. - **Compensating controls:** a bounded window (e.g. 30 days); a startup **AUDIT** line records the override. -- **Still refused:** a **PHI** instance with an unbounded PHI-body window refuses under **strict enforcement** - (`enforcement = enforce`, the default) unless `allow_keeping_phi_indefinitely = true` (which downgrades the - refusal to a loud audited warning); at `enforcement = warn` a PHI instance auto-bounds each unset window to - 30 days. +- **Still refused:** a **PHI** instance whose PHI-body window is set **explicitly to `0`** refuses under + **strict enforcement** (`enforcement = enforce`, the default) unless `allow_keeping_phi_indefinitely = true` + (which downgrades the refusal to a loud audited warning). This bullet used to say the 30-day auto-bound of + an *unset* window happened only at `enforcement = warn`. It happens on **both** dials, so the refusal above + is reached only by an explicit `0` — or by the opt-out itself, which suppresses the auto-bound and therefore + leaves an unset window unbounded. ### `allow_single_factor_admin_when_exposed = true` — lift the strict-enforcement single-factor-admin refusal - **What you lose:** on a **PHI** instance under **strict enforcement** (`enforcement = enforce`, the default) @@ -295,7 +297,8 @@ trail. (`--allow-insecure-bind` / `MEFOR_ALLOW_INSECURE_TLS`) are **honoured** again. This reproduces the historical **non-production** PHI behaviour on a box that is otherwise strict-by-default: the cleartext off-box bind, open-egress, and single-factor-admin-at-exposure refusals downgrade to loud audited warnings, - and an unset PHI retention window auto-bounds to 30 days rather than refusing. Named once by + and an explicitly-zeroed PHI retention window warns rather than refusing. (The 30-day auto-bound of an + **unset** window is not part of this dial — it applies under `enforce` too.) Named once by `security_loosenings()` and in `GET /security/posture`. - **When acceptable:** a PHI **staging / pre-prod** box that must mirror production's *config* (so the encryption / egress / retention paths are exercised, not first met in production) but is deliberately run at diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 35770494..51d4f218 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -880,14 +880,23 @@ allow-listed provider values only, one session per form POST (the AD role-resync effect fires once at login, never per navigation), MFA stays delegated to the directory. `require_mfa` defaults **on** (BACKLOG #187 — secure-by-default, including the loopback bind; the -documented org opt-out is `[auth].require_mfa = false`). The exposure gate now guards the **explicit +documented org opt-out is `[security].require_mfa = false` — the `[auth]` spelling of this key is +**rejected at load** and `serve` exits 2 naming the replacement). The exposure gate now guards the **explicit opt-out**: when the API is bound **off-loopback** with `require_mfa` *turned off*, `serve` makes the posture explicit at startup — it **refuses to start** on a **production PHI** instance and **warns** on a non-production PHI instance (a synthetic instance stays quiet), mirroring the keyless-store and open-egress startup gates. So an exposed PHI deployment can't silently run the Administrator interface -single-factor. `require_mfa` is safe to keep on even on an **AD-only** deployment — it gates only -**local** Administrator accounts (AD/Kerberos MFA stays delegated to the directory), so an operator who -opts out at exposure simply re-enables `[auth].require_mfa=true` (or keeps the bind on loopback). +single-factor. `require_mfa` is safe to keep on for an **AD-only** deployment's *directory* users: +AD/Kerberos identities are exempt under either `require_mfa_scope` value, their factor delegated to the +directory. An earlier revision of this sentence said it "gates only **local** Administrator accounts"; +that was wrong. Under the shipped `require_mfa_scope = "every_local_account"` it covers **every** local +account, which on an AD-only deployment still means the local bootstrap admin and any local service +accounts — a non-interactive local bearer-token account becomes MFA-pending and cannot enrol +unattended. **That is a decision a deploying site must make before first start:** either such an +account becomes an AD principal, or the scope is set to `administrators`. An operator who opts out at +exposure re-enables `[security].require_mfa = true` (or keeps the bind on loopback). +[CONFIGURATION.md](CONFIGURATION.md) `[security].require_mfa_scope` is the authority on the two +remedies and on why mTLS is not a third. ### Administrative-interface defense-in-depth (WP-L3-13, ASVS 8.4.2) @@ -1144,7 +1153,7 @@ one-to-one — that is why the bind/exposure posture occupies two rows and the A | New client IP during a session | this request's address vs `session.client` | knob on **and** a session exists, is unrevoked, has an anchor, and the two are not the same host (both-loopback counts as one host) | **CHALLENGE** — force a fresh step-up; first sighting also writes `auth.admin_action_new_ip` + an out-of-band notice; repeats WARNING-log only. **Never** an RBAC deny | **off** | `[auth].admin_new_ip_step_up` | | Credential recency | age of `session.reauth_at` | `now − reauth_at > step_up_max_age_seconds`, or `reauth_at is None` | **DENY** 403 + `X-Step-Up-Required: 1` (console: 303 → `/ui/reauth`) | 300 s | `[auth].step_up_max_age_seconds` | | Action-bound step-up grant | a single-use grant minted only by `reauth(purpose=…)`, on the **monotonic** clock | no unconsumed grant for this route's action | **DENY** 403 + `X-Step-Up-Required` + `X-Step-Up-Action: `; opting out falls back to the session window | on | `[auth].require_action_step_up` | -| MFA state | `session.mfa_verified_at` × factor enrollment × account roles | non-LOCAL account → never required; LOCAL + enrolled → always; LOCAL + un-enrolled → required when the knob is on **and** the account holds Administrator | **DENY** 403 + `X-MFA-Required: 1` at the step-up boundary | on | `[auth].require_mfa` | +| MFA state | `session.mfa_verified_at` × factor enrollment × account roles | AD account → never required here (directory MFA is delegated); LOCAL + enrolled → always, whatever the scope says; LOCAL + un-enrolled → required when the knob is on **and** the scope covers the account — **`every_local_account` by default**, i.e. every local account, or the Administrator role only under `administrators` | **DENY** 403 + `X-MFA-Required: 1` on **every** authorized route — an **access gate**, not only a step-up gate; the console twin is a 303 to `/ui/mfa`, with the account and factor-enrolment routes exempt so an un-enrolled user is not stranded. An earlier revision of this row said Administrator-only and step-up-boundary-only; both were wrong | on; scope `every_local_account` | `[security].require_mfa`, `[security].require_mfa_scope` (the `[auth]` spellings are rejected at load) | | Identity provider — local credential rotation | `identity.auth_provider` | the provider is AD (the credential is the directory's, not the engine's) | **DENY** `POST /me/password` with **400**; the step-up re-proof for that identity becomes a **live directory re-bind** instead of a local hash compare, so a disabled AD account cannot refresh its window, and the engine MFA gate never fires for it | n/a | `[auth].ad_enabled` | | Authentication ambience | how the session was minted | browser Kerberos SSO and the OIDC callback mint with `seed_reauth=False` | **CHALLENGE** — the session is born **without** step-up freshness, so its first sensitive action forces an explicit credential step-up (the *second* signal in this table whose action is a challenge rather than a hard decision) | n/a | (by design) | | Session age | `created_at` / `last_used_at` / `expires_at` vs wall clock, on **every** request | idle > 30 min; past the absolute expiry (12 h, or a tighter signature-verified federated `id_token.exp` cap); or a **backward** wall-clock step (NTP step-back, VM snapshot revert) | **DENY** — the session is revoked in the store, then 401. The idle clock is refreshed only by user-driven requests, so a background poll cannot keep a session alive | 30 min / 12 h | `[security].sign_out_after_idle_minutes`, `max_session_hours` (the ADR 0118 homes; `[auth].session_idle_timeout_minutes` / `session_absolute_hours` are the retired aliases), plus `[auth].oidc_session_max_hours` for a tighter federated cap | @@ -1445,9 +1454,32 @@ MFA step-up is now built (WP-14 native TOTP); a web console banner for the feed Local passwords follow an **ASVS 5.0-aligned** policy (WP-3): **min length 15**, **no mandatory character-class composition** (the `require_*` class flags are opt-in, default off — ASVS forbids mandatory composition), plus **offline breached/common-password screening** (a bundled top-10k list, -no live HIBP call) and a small **context-word deny-list** (app/vendor/HL7 terms like `messagefoundry`, -`mefor`, `hl7`, `corepoint`). Enforced identically on create-user and change-password; tune via -`[auth]` (see [CONFIGURATION.md](CONFIGURATION.md)). AD passwords are governed by Active Directory. +no live HIBP call) and a fixed **context-word deny-list**, enumerated in full below. Enforced +identically on create-user and change-password; tune via `[auth]` (see +[CONFIGURATION.md](CONFIGURATION.md)). AD passwords are governed by Active Directory. + +**The context-word deny-list, in full.** A local password is refused if it *contains* any of these +twelve terms as a case-insensitive substring, anywhere in the value — not only as a prefix, and not +only as a whole word: + +`messagefoundry`, `mefor`, `mllp`, `hl7`, `corepoint`, `mirth`, `rhapsody`, `changeme`, `bootstrap`, +`admin`, `administrator`, `password` + +An earlier revision of this page described the list as "app/vendor/HL7 terms" and showed four of the +twelve as examples. That description was wrong in a way a reader could act on: five members — +`changeme`, `bootstrap`, `admin`, `administrator`, `password` — are generic credential words with no +connection to this application, to a vendor, or to HL7, so a passphrase chosen on the strength of the +old sentence could still be refused with no indication of which rule fired. The list above is the +whole of it, mirrored from `CONTEXT_WORDS` in +[`auth/policy.py`](../messagefoundry/auth/policy.py); the code is the authority if the two diverge. + +**What a deploying site can and cannot tune here.** `password_check_context` is a whole-list on/off +switch, on by default. There is **no** setting that adds a site's own terms — its hospital +abbreviation, a partner or product name, the local domain — and none that removes a member whose +substring collides with a legitimate local word. A site that wants wider coverage supplies it through +`password_breach_corpus_file` below, which answers a different question: that corpus is matched +against the **whole** password, so a term added there is refused only when it *is* the password, never +when it appears inside a longer passphrase. Two further screens (ASVS 6.2.11 / 6.2.12), both on by default and fully offline: @@ -1467,7 +1499,7 @@ service-identity plane. | Pathway | Factor | Brute-force defense | Notes | |---|---|---|---| -| **Local** (argon2id) | **password** (argon2id) **plus an engine second factor** — RFC 6238 TOTP, single-use recovery codes, or a WebAuthn/FIDO2 passkey. Since ASVS 6.3.3 that factor is an **access gate, not merely a step-up boundary**: an MFA-pending session is refused on *every* authorized route with `X-MFA-Required: 1` (browser sessions are confined to `/ui/mfa`). It binds any local account that has enrolled a factor, plus every account `[auth].require_mfa_scope` covers — **`every_local_account` by default** (`[auth].require_mfa` defaults **on**). Set the scope to `administrators` for the pre-6.3.3 posture, in which a non-admin, un-enrolled local session is **password-only end to end**. Caveat: a passkey is asserted at `user_verification=preferred`, so for a passkey-only account the second factor may be **device possession alone** | **per-account lockout** (5/15 min), fed by **both** the password and the TOTP/recovery leg + breach/context policy + the per-IP **and** global sign-in window | the only pathway the engine itself can lock out; the only one with a phishing-resistant factor | +| **Local** (argon2id) | **password** (argon2id) **plus an engine second factor** — RFC 6238 TOTP, single-use recovery codes, or a WebAuthn/FIDO2 passkey. That factor is an **access gate, not merely a step-up boundary**: an MFA-pending session is refused on *every* authorized route with `X-MFA-Required: 1`, and a browser session is **redirected** to `/ui/mfa` — *not* confined to it, as an earlier revision of this cell said, because the account and factor-enrolment routes are declared MFA-pending-exempt, so a user with no factor yet enrols at `/ui/account`. It binds any local account that has enrolled a factor, plus every account `[security].require_mfa_scope` covers — **`every_local_account` by default** (`[security].require_mfa` defaults **on**; both keys are rejected under `[auth]` and fail the start). Set the scope to `administrators` for the earlier, narrower posture, in which a non-admin, un-enrolled local session is **password-only end to end**. Caveat: a passkey is asserted at `user_verification=preferred`, so for a passkey-only account the second factor may be **device possession alone** | **per-account lockout** (5/15 min), fed by **both** the password and the TOTP/recovery leg + breach/context policy + the per-IP **and** global sign-in window | the only pathway the engine itself can lock out; the only one with a phishing-resistant factor | | **AD** (LDAP simple-bind, LDAPS by default) | password, verified by a bind **as the user** against the DC; MFA is **delegated and unverifiable at the engine** — the simple-bind call site issues the session MFA-satisfied under the owner-signed delegated-directory relaxation, regardless of what the directory enforced, and no `amr`-equivalent evidence is received. The grant is now a per-mechanism argument rather than a blanket literal, so the federated leg can differ (see OIDC) | the **directory's** lockout/complexity policy; engine-side, the per-IP **and** global sign-in window (`[auth].login_rate_limit_enabled`, default on — **off leaves this pathway with no engine-side control at all**) — and **no** engine per-account lockout | password strength + lockout are the AD domain's responsibility. LDAPS is the default, not a structural guarantee: `[auth].ad_allow_insecure_ldap` opts into a plain bind, and `ad_tls_verify=false` is refused at startup unless the `MEFOR_ALLOW_INSECURE_TLS` dev escape is set | | **Kerberos / SPNEGO** | domain ticket; MFA is **delegated and unverifiable at the engine** — the session is issued MFA-satisfied and no `amr`-equivalent evidence is received | the **domain's** controls; engine-side, the sign-in window on the token-bearing leg (`[auth].login_rate_limit_enabled`, default on — **off leaves this pathway with no engine-side control at all**; the RFC 4559 challenge leg is deliberately unthrottled either way) | experimental, off by default, **single-leg — no mutual authentication**, channel binding deliberately un-enforced. The browser leg (`GET /ui/sso`) mints with no step-up window, so the first sensitive action forces a step-up; the JSON `POST /auth/negotiate` seeds it | | **OIDC federation** (browser only, hybrid AD-backed) | IdP-asserted, gated on a **signature-verified** `amr`/`acr` claim (`[auth].oidc_require_mfa_claim` defaults **on**) — an assertion, not a proof | no engine credential to guess, so no per-account lockout; both legs (`/ui/oidc/start`, `/ui/oidc/callback`) charge the sign-in window (`[auth].login_rate_limit_enabled`, default on — **off leaves this pathway with no engine-side control at all**, though the bounded pending-flow cache still caps concurrent start legs), plus the IdP's own lockout | hybrid-only: a federated principal with no on-prem AD object is refused. Roles come from LDAP, never from a token claim. When `[auth].oidc_username_strip_domain` is on (default), the claim's UPN suffix must match `oidc_allowed_username_domains` (or `[auth].ad_domain`); with stripping **off** the claim is used verbatim and no suffix check applies. The session's absolute lifetime is capped at the verified `id_token.exp`; minted with no step-up window | @@ -1528,15 +1560,28 @@ delegated pathways rather than narrowing it, and an operator turning it off must lockout policy carrying the whole load. OIDC has no engine credential to lock out; the mTLS plane has no guessable secret at all, so no rate limit or lockout applies to it. **A genuine second factor is built for local accounts only** — TOTP (WP-14) *and* WebAuthn passkeys (WP-14b), the latter being the only -phishing-resistant factor shipped; `[auth].require_mfa` defaults on but scopes to local Administrator -accounts, and it is enforced at the **step-up boundary, not as an access gate** (see -[Multi-factor authentication](#multi-factor-authentication-totp-wp-14) and the separate 6.3.3 scoring). +phishing-resistant factor shipped; `[security].require_mfa` defaults **on** and its shipped scope is +**`every_local_account`** rather than the Administrator role, and it is enforced as an **access gate, not +only at the step-up boundary** — an MFA-pending session is refused on every authorized route. An earlier +revision of this sentence asserted the opposite on both counts and named the `[auth]` keys the loader +rejects; it also contradicted the Local row of the table above, which was right (see +[Multi-factor authentication](#multi-factor-authentication-totp-wp-14)). AD/Kerberos MFA is delegated to the directory; OIDC's is asserted by the IdP and gated on a -signature-verified claim. **The consequence, stated plainly:** every directory pathway satisfies the -engine's MFA gates without an engine-verified factor — `_complete_ad_login` mints AD, Kerberos and OIDC -sessions `mfa_verified=True` unconditionally — so a *password* on the AD pathway reaches the same PHI -surface as a passkey-backed local Administrator. OIDC is the only delegated pathway that carries any -engine-side evidence at all (the signature-verified `amr`/`acr` gate); AD and Kerberos carry none. +signature-verified claim. **The consequence, stated plainly:** the AD and Kerberos pathways satisfy the +engine's MFA gates without an engine-verified factor, so a *password* on the AD pathway reaches the same +PHI surface as a passkey-backed local Administrator. The mechanism is a **per-mechanism argument**, not a +blanket literal: `mfa_verified` is a keyword parameter of `_complete_ad_login`, and the simple-bind and +Kerberos legs pass `True` under the signed delegated-directory relaxation while the federated leg passes +`[auth].oidc_require_mfa_claim` itself — on by default, and reached only after the claim gate has already +refused any token carrying no configured `amr`/`acr`, so the grant there is engine-verified rather than +assumed. Turn that setting off and the federated session mints **un**verified, and `mfa_satisfied` +refuses it while `[security].require_mfa` is on (the default). An earlier revision of this sentence said +`_complete_ad_login` mints all three `mfa_verified=True` unconditionally; that was wrong about the +mechanism and contradicted the OIDC row of the table above. OIDC therefore remains the only delegated +pathway carrying engine-side evidence at all; AD and Kerberos carry none, and **closing that gap is the +deploying site's job, in the directory** — the engine accepts whatever the directory asserts, so the +domain's own MFA policy (Entra Conditional Access or an MFA proxy) is the only control over those two +pathways. ## Brute-force & abuse protection diff --git a/tests/test_docs_security_pathways.py b/tests/test_docs_security_pathways.py index 18bd3936..eae1b01c 100644 --- a/tests/test_docs_security_pathways.py +++ b/tests/test_docs_security_pathways.py @@ -218,7 +218,12 @@ def test_companion_table_covers_the_remaining_strength_dimensions() -> None: True, "`[auth].oidc_require_mfa_claim` defaults **on**", ), - (AuthSettings, "require_mfa", True, "`[auth].require_mfa` defaults **on**"), + # The MODEL field stays `AuthSettings.require_mfa` (the internal desugared field), but the + # OPERATOR-FACING key is `[security].require_mfa`: `("auth", "require_mfa")` is in + # `_RELOCATED_TO_SECURITY`, so the `[auth]` spelling raises at load and `serve` exits 2. The + # rendered token has to quote the key an operator can actually set, or this guard pins the + # documentation to a config that cannot start. + (AuthSettings, "require_mfa", True, "`[security].require_mfa` defaults **on**"), ( AuthSettings, "ad_session_recheck_seconds", diff --git a/tests/test_key_usage_scope_inventory.py b/tests/test_key_usage_scope_inventory.py index e4cca75f..9aa46ffe 100644 --- a/tests/test_key_usage_scope_inventory.py +++ b/tests/test_key_usage_scope_inventory.py @@ -50,6 +50,10 @@ "OIDC IdP JWKS verification keys", "OIDC IdP TLS trust anchor", "Cert tooling", + "Outbound SMTP transport TLS", + "Inbound XML-DSig verification", + "SFTP transport", + "SMART Backend Services client assertion", } )