From 9b01b34be3fd95119290161176243debc07d1519 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Mon, 17 Aug 2026 15:38:00 -0500 Subject: [PATCH 1/5] docs(security): enumerate the password context deny-list instead of sampling it Both places that described the local-password context check called it "app/vendor/HL7 terms" and showed four of the twelve enforced terms behind a "like" / "e.g." hedge. The category was wrong, not just partial: five of the twelve members are changeme, bootstrap, admin, administrator and password, which belong to no vendor and to no protocol. An operator sizing a password standard against that sentence would expect Bootstrap-Winter-2026! to pass, and it is refused with no indication of which rule fired. SECURITY.md now lists all twelve verbatim from CONTEXT_WORDS in auth/policy.py, states that the match is case-insensitive SUBSTRING containment rather than equality or a prefix, and names the retired description rather than quietly widening the sample. It also records what the knob cannot do: password_check_context is whole-list on/off, no setting adds or removes a term, and password_breach_corpus_file is not a substitute because it matches the whole password only. CONFIGURATION.md's settings row carries the same correction and links rather than repeating the list, so the two cannot drift apart again. --- docs/CONFIGURATION.md | 2 +- docs/SECURITY.md | 29 ++++++++++++++++++++++++++--- 2 files changed, 27 insertions(+), 4 deletions(-) diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 302419a7..e747e9b8 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -562,7 +562,7 @@ document ([SECURITY-DOCS-POLICY.md](SECURITY-DOCS-POLICY.md)). | `password_min_length` | int | 15 | local-password policy — ASVS 5.0-aligned, length-first | | `password_require_uppercase` / `password_require_lowercase` / `password_require_digit` / `password_require_symbol` | bool | `false` | character classes — **opt-in**, each independently (ASVS 5.0 forbids mandatory composition); turn one on only for a legacy standard that still mandates it | | `password_check_breached` | bool | `true` | reject known common/breached passwords against a bundled offline top-10k list (no live HIBP call) | -| `password_check_context` | bool | `true` | reject passwords containing app/vendor/HL7 terms (e.g. `messagefoundry`, `mefor`, `hl7`, `corepoint`) | +| `password_check_context` | bool | `true` | reject a local password that **contains** any deny-list term — a case-insensitive substring test, anywhere in the value, not a whole-word or prefix match. The **twelve** terms are listed in full in [SECURITY.md](SECURITY.md) "Password policy"; an earlier revision of this row called them "app/vendor/HL7 terms" and gave four examples, which mis-stated the rule (five of the twelve are generic credential words unrelated to this application or to HL7). The list is fixed in code (`CONTEXT_WORDS` in [`auth/policy.py`](../messagefoundry/auth/policy.py)): **this flag turns the whole check on or off, and no setting adds or removes a term**, so a site needing its own vocabulary uses `password_breach_corpus_file` below — which matches the *whole* password, never a substring | | `password_check_username` | bool | `true` | reject a password containing the user's **own username** (ASVS 6.2.11) | | `password_breach_corpus_file` | path | — | optional path to a **larger offline breach corpus** that augments the bundled top-10k list (ASVS 6.2.12): a plaintext list **or** an HIBP-style SHA-1 hash export (`HASH[:count]` lines, auto-detected). Fully offline — still no live HIBP call. Use a curated subset, not the full ~40 GB HIBP set (it is loaded into memory). A path, not a secret | | `lockout_threshold` | int | 5 | failed logins before lock (per account) | diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 35770494..f0ddb0b6 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -1445,9 +1445,32 @@ MFA step-up is now built (WP-14 native TOTP); a web console banner for the feed Local passwords follow an **ASVS 5.0-aligned** policy (WP-3): **min length 15**, **no mandatory character-class composition** (the `require_*` class flags are opt-in, default off — ASVS forbids mandatory composition), plus **offline breached/common-password screening** (a bundled top-10k list, -no live HIBP call) and a small **context-word deny-list** (app/vendor/HL7 terms like `messagefoundry`, -`mefor`, `hl7`, `corepoint`). Enforced identically on create-user and change-password; tune via -`[auth]` (see [CONFIGURATION.md](CONFIGURATION.md)). AD passwords are governed by Active Directory. +no live HIBP call) and a fixed **context-word deny-list**, enumerated in full below. Enforced +identically on create-user and change-password; tune via `[auth]` (see +[CONFIGURATION.md](CONFIGURATION.md)). AD passwords are governed by Active Directory. + +**The context-word deny-list, in full.** A local password is refused if it *contains* any of these +twelve terms as a case-insensitive substring, anywhere in the value — not only as a prefix, and not +only as a whole word: + +`messagefoundry`, `mefor`, `mllp`, `hl7`, `corepoint`, `mirth`, `rhapsody`, `changeme`, `bootstrap`, +`admin`, `administrator`, `password` + +An earlier revision of this page described the list as "app/vendor/HL7 terms" and showed four of the +twelve as examples. That description was wrong in a way a reader could act on: five members — +`changeme`, `bootstrap`, `admin`, `administrator`, `password` — are generic credential words with no +connection to this application, to a vendor, or to HL7, so a passphrase chosen on the strength of the +old sentence could still be refused with no indication of which rule fired. The list above is the +whole of it, mirrored from `CONTEXT_WORDS` in +[`auth/policy.py`](../messagefoundry/auth/policy.py); the code is the authority if the two diverge. + +**What a deploying site can and cannot tune here.** `password_check_context` is a whole-list on/off +switch, on by default. There is **no** setting that adds a site's own terms — its hospital +abbreviation, a partner or product name, the local domain — and none that removes a member whose +substring collides with a legitimate local word. A site that wants wider coverage supplies it through +`password_breach_corpus_file` below, which answers a different question: that corpus is matched +against the **whole** password, so a term added there is refused only when it *is* the password, never +when it appears inside a longer passphrase. Two further screens (ASVS 6.2.11 / 6.2.12), both on by default and fully offline: From e58bfa9481120800f5559b7ed6294d3ec17407dd Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Mon, 17 Aug 2026 15:39:08 -0500 Subject: [PATCH 2/5] docs(security): correct the MFA scope, enforcement point and setting names Four claims about the second-factor requirement were refuted by the shipped code, and the document contradicted itself on two of them. Scope. The pathway-strength paragraph said require_mfa "scopes to local Administrator accounts". The shipped default is require_mfa_scope = "every_local_account" (config/settings.py), whose own comment records that the default widens the gate past the Administrator role. The Local pathway row sixty lines earlier already said this correctly. Enforcement point. The same paragraph, and the MFA-state row of the enforcement table, placed the refusal "at the step-up boundary, not as an access gate". api/security.py evaluates it above the permission loop and refuses every authorized route with 403 + X-MFA-Required: 1. Setting names. [auth].require_mfa and [auth].require_mfa_scope are in _RELOCATED_TO_SECURITY; the loader raises on them from file and environment alike and serve exits 2. The documented opt-out therefore named a config that cannot start. All sites now name the [security] keys and say the old spelling is rejected. [auth].oidc_require_mfa_claim is NOT relocated and keeps its spelling. Delegated pathways. "_complete_ad_login mints AD, Kerberos and OIDC sessions mfa_verified=True unconditionally" was wrong about the mechanism: it is a keyword argument decided per mechanism, and the federated leg passes oidc_require_mfa_claim, reached only after the claim gate has refused a token carrying no configured amr/acr. Under the shipped default the outcome is the same, which is why the wrong mechanism survived; turning the claim gate off does change it, and the old text implied otherwise. Two smaller fixes in the same cells: browser sessions are redirected to /ui/mfa, not confined to it (the account and enrolment routes are MFA-pending-exempt, which is what stops a zero-factor user being stranded), and an AD principal is exempt even with a factor enrolled, so the "always required once enrolled" clause is scoped to local accounts. PHI.md's control row asserted the same retired step-up/Administrator framing and is corrected with it, so the two documents agree. tests/test_docs_security_pathways.py pinned the literal "[auth].require_mfa defaults on" inside the section being corrected. The model field is unchanged (AuthSettings.require_mfa is the internal desugared field); only the rendered operator-facing key moves to [security], because the guard was pinning the documentation to a key the loader rejects. --- docs/PHI.md | 2 +- docs/SECURITY.md | 50 ++++++++++++++++++++-------- tests/test_docs_security_pathways.py | 7 +++- 3 files changed, 43 insertions(+), 16 deletions(-) diff --git a/docs/PHI.md b/docs/PHI.md index 9e7fd2e9..a40b3c56 100644 --- a/docs/PHI.md +++ b/docs/PHI.md @@ -1302,7 +1302,7 @@ separate follow-up. | Item | Closes | Maps to | Effort | |---|---|---|---| | **P2-1** TLS on the engine API | Tokens + PHI cleartext over the network | §164.312(e) · SC-8 | M | -| **P2-2** MFA for console/API auth — ✅ **Built (WP-14, native TOTP, local accounts)** | Single-factor auth (mitigated for local accounts: `[auth].require_mfa` gates **step-up / sensitive admin operations** for the Administrator role — not every PHI read; AD MFA delegated) | §164.312(d) · IA-2(1) (NPRM-mandated) | M–L | +| **P2-2** MFA for console/API auth — ✅ **Built (WP-14, native TOTP, local accounts)** | Single-factor auth (mitigated for local accounts: `[security].require_mfa` is an **access gate on every authorized route**, and its shipped `require_mfa_scope` is `every_local_account`, not the Administrator role alone; AD MFA delegated) | §164.312(d) · IA-2(1) (NPRM-mandated) | M–L | | **P2-3** Network-segmentation guidance + periodic integrity checks | Lateral movement; tamper detection | §164.312(c) · SC-7/SI-7 | S–M | | **P2-4** Strict-parse CPU/time budget on the hl7apy path | Malformed input pinning a worker — message size/segment caps are built, but the opt-in strict parse itself has no time bound | NIST SC-5 (DoS; not a §164.312 safeguard) | S | diff --git a/docs/SECURITY.md b/docs/SECURITY.md index f0ddb0b6..51d4f218 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -880,14 +880,23 @@ allow-listed provider values only, one session per form POST (the AD role-resync effect fires once at login, never per navigation), MFA stays delegated to the directory. `require_mfa` defaults **on** (BACKLOG #187 — secure-by-default, including the loopback bind; the -documented org opt-out is `[auth].require_mfa = false`). The exposure gate now guards the **explicit +documented org opt-out is `[security].require_mfa = false` — the `[auth]` spelling of this key is +**rejected at load** and `serve` exits 2 naming the replacement). The exposure gate now guards the **explicit opt-out**: when the API is bound **off-loopback** with `require_mfa` *turned off*, `serve` makes the posture explicit at startup — it **refuses to start** on a **production PHI** instance and **warns** on a non-production PHI instance (a synthetic instance stays quiet), mirroring the keyless-store and open-egress startup gates. So an exposed PHI deployment can't silently run the Administrator interface -single-factor. `require_mfa` is safe to keep on even on an **AD-only** deployment — it gates only -**local** Administrator accounts (AD/Kerberos MFA stays delegated to the directory), so an operator who -opts out at exposure simply re-enables `[auth].require_mfa=true` (or keeps the bind on loopback). +single-factor. `require_mfa` is safe to keep on for an **AD-only** deployment's *directory* users: +AD/Kerberos identities are exempt under either `require_mfa_scope` value, their factor delegated to the +directory. An earlier revision of this sentence said it "gates only **local** Administrator accounts"; +that was wrong. Under the shipped `require_mfa_scope = "every_local_account"` it covers **every** local +account, which on an AD-only deployment still means the local bootstrap admin and any local service +accounts — a non-interactive local bearer-token account becomes MFA-pending and cannot enrol +unattended. **That is a decision a deploying site must make before first start:** either such an +account becomes an AD principal, or the scope is set to `administrators`. An operator who opts out at +exposure re-enables `[security].require_mfa = true` (or keeps the bind on loopback). +[CONFIGURATION.md](CONFIGURATION.md) `[security].require_mfa_scope` is the authority on the two +remedies and on why mTLS is not a third. ### Administrative-interface defense-in-depth (WP-L3-13, ASVS 8.4.2) @@ -1144,7 +1153,7 @@ one-to-one — that is why the bind/exposure posture occupies two rows and the A | New client IP during a session | this request's address vs `session.client` | knob on **and** a session exists, is unrevoked, has an anchor, and the two are not the same host (both-loopback counts as one host) | **CHALLENGE** — force a fresh step-up; first sighting also writes `auth.admin_action_new_ip` + an out-of-band notice; repeats WARNING-log only. **Never** an RBAC deny | **off** | `[auth].admin_new_ip_step_up` | | Credential recency | age of `session.reauth_at` | `now − reauth_at > step_up_max_age_seconds`, or `reauth_at is None` | **DENY** 403 + `X-Step-Up-Required: 1` (console: 303 → `/ui/reauth`) | 300 s | `[auth].step_up_max_age_seconds` | | Action-bound step-up grant | a single-use grant minted only by `reauth(purpose=…)`, on the **monotonic** clock | no unconsumed grant for this route's action | **DENY** 403 + `X-Step-Up-Required` + `X-Step-Up-Action: `; opting out falls back to the session window | on | `[auth].require_action_step_up` | -| MFA state | `session.mfa_verified_at` × factor enrollment × account roles | non-LOCAL account → never required; LOCAL + enrolled → always; LOCAL + un-enrolled → required when the knob is on **and** the account holds Administrator | **DENY** 403 + `X-MFA-Required: 1` at the step-up boundary | on | `[auth].require_mfa` | +| MFA state | `session.mfa_verified_at` × factor enrollment × account roles | AD account → never required here (directory MFA is delegated); LOCAL + enrolled → always, whatever the scope says; LOCAL + un-enrolled → required when the knob is on **and** the scope covers the account — **`every_local_account` by default**, i.e. every local account, or the Administrator role only under `administrators` | **DENY** 403 + `X-MFA-Required: 1` on **every** authorized route — an **access gate**, not only a step-up gate; the console twin is a 303 to `/ui/mfa`, with the account and factor-enrolment routes exempt so an un-enrolled user is not stranded. An earlier revision of this row said Administrator-only and step-up-boundary-only; both were wrong | on; scope `every_local_account` | `[security].require_mfa`, `[security].require_mfa_scope` (the `[auth]` spellings are rejected at load) | | Identity provider — local credential rotation | `identity.auth_provider` | the provider is AD (the credential is the directory's, not the engine's) | **DENY** `POST /me/password` with **400**; the step-up re-proof for that identity becomes a **live directory re-bind** instead of a local hash compare, so a disabled AD account cannot refresh its window, and the engine MFA gate never fires for it | n/a | `[auth].ad_enabled` | | Authentication ambience | how the session was minted | browser Kerberos SSO and the OIDC callback mint with `seed_reauth=False` | **CHALLENGE** — the session is born **without** step-up freshness, so its first sensitive action forces an explicit credential step-up (the *second* signal in this table whose action is a challenge rather than a hard decision) | n/a | (by design) | | Session age | `created_at` / `last_used_at` / `expires_at` vs wall clock, on **every** request | idle > 30 min; past the absolute expiry (12 h, or a tighter signature-verified federated `id_token.exp` cap); or a **backward** wall-clock step (NTP step-back, VM snapshot revert) | **DENY** — the session is revoked in the store, then 401. The idle clock is refreshed only by user-driven requests, so a background poll cannot keep a session alive | 30 min / 12 h | `[security].sign_out_after_idle_minutes`, `max_session_hours` (the ADR 0118 homes; `[auth].session_idle_timeout_minutes` / `session_absolute_hours` are the retired aliases), plus `[auth].oidc_session_max_hours` for a tighter federated cap | @@ -1490,7 +1499,7 @@ service-identity plane. | Pathway | Factor | Brute-force defense | Notes | |---|---|---|---| -| **Local** (argon2id) | **password** (argon2id) **plus an engine second factor** — RFC 6238 TOTP, single-use recovery codes, or a WebAuthn/FIDO2 passkey. Since ASVS 6.3.3 that factor is an **access gate, not merely a step-up boundary**: an MFA-pending session is refused on *every* authorized route with `X-MFA-Required: 1` (browser sessions are confined to `/ui/mfa`). It binds any local account that has enrolled a factor, plus every account `[auth].require_mfa_scope` covers — **`every_local_account` by default** (`[auth].require_mfa` defaults **on**). Set the scope to `administrators` for the pre-6.3.3 posture, in which a non-admin, un-enrolled local session is **password-only end to end**. Caveat: a passkey is asserted at `user_verification=preferred`, so for a passkey-only account the second factor may be **device possession alone** | **per-account lockout** (5/15 min), fed by **both** the password and the TOTP/recovery leg + breach/context policy + the per-IP **and** global sign-in window | the only pathway the engine itself can lock out; the only one with a phishing-resistant factor | +| **Local** (argon2id) | **password** (argon2id) **plus an engine second factor** — RFC 6238 TOTP, single-use recovery codes, or a WebAuthn/FIDO2 passkey. That factor is an **access gate, not merely a step-up boundary**: an MFA-pending session is refused on *every* authorized route with `X-MFA-Required: 1`, and a browser session is **redirected** to `/ui/mfa` — *not* confined to it, as an earlier revision of this cell said, because the account and factor-enrolment routes are declared MFA-pending-exempt, so a user with no factor yet enrols at `/ui/account`. It binds any local account that has enrolled a factor, plus every account `[security].require_mfa_scope` covers — **`every_local_account` by default** (`[security].require_mfa` defaults **on**; both keys are rejected under `[auth]` and fail the start). Set the scope to `administrators` for the earlier, narrower posture, in which a non-admin, un-enrolled local session is **password-only end to end**. Caveat: a passkey is asserted at `user_verification=preferred`, so for a passkey-only account the second factor may be **device possession alone** | **per-account lockout** (5/15 min), fed by **both** the password and the TOTP/recovery leg + breach/context policy + the per-IP **and** global sign-in window | the only pathway the engine itself can lock out; the only one with a phishing-resistant factor | | **AD** (LDAP simple-bind, LDAPS by default) | password, verified by a bind **as the user** against the DC; MFA is **delegated and unverifiable at the engine** — the simple-bind call site issues the session MFA-satisfied under the owner-signed delegated-directory relaxation, regardless of what the directory enforced, and no `amr`-equivalent evidence is received. The grant is now a per-mechanism argument rather than a blanket literal, so the federated leg can differ (see OIDC) | the **directory's** lockout/complexity policy; engine-side, the per-IP **and** global sign-in window (`[auth].login_rate_limit_enabled`, default on — **off leaves this pathway with no engine-side control at all**) — and **no** engine per-account lockout | password strength + lockout are the AD domain's responsibility. LDAPS is the default, not a structural guarantee: `[auth].ad_allow_insecure_ldap` opts into a plain bind, and `ad_tls_verify=false` is refused at startup unless the `MEFOR_ALLOW_INSECURE_TLS` dev escape is set | | **Kerberos / SPNEGO** | domain ticket; MFA is **delegated and unverifiable at the engine** — the session is issued MFA-satisfied and no `amr`-equivalent evidence is received | the **domain's** controls; engine-side, the sign-in window on the token-bearing leg (`[auth].login_rate_limit_enabled`, default on — **off leaves this pathway with no engine-side control at all**; the RFC 4559 challenge leg is deliberately unthrottled either way) | experimental, off by default, **single-leg — no mutual authentication**, channel binding deliberately un-enforced. The browser leg (`GET /ui/sso`) mints with no step-up window, so the first sensitive action forces a step-up; the JSON `POST /auth/negotiate` seeds it | | **OIDC federation** (browser only, hybrid AD-backed) | IdP-asserted, gated on a **signature-verified** `amr`/`acr` claim (`[auth].oidc_require_mfa_claim` defaults **on**) — an assertion, not a proof | no engine credential to guess, so no per-account lockout; both legs (`/ui/oidc/start`, `/ui/oidc/callback`) charge the sign-in window (`[auth].login_rate_limit_enabled`, default on — **off leaves this pathway with no engine-side control at all**, though the bounded pending-flow cache still caps concurrent start legs), plus the IdP's own lockout | hybrid-only: a federated principal with no on-prem AD object is refused. Roles come from LDAP, never from a token claim. When `[auth].oidc_username_strip_domain` is on (default), the claim's UPN suffix must match `oidc_allowed_username_domains` (or `[auth].ad_domain`); with stripping **off** the claim is used verbatim and no suffix check applies. The session's absolute lifetime is capped at the verified `id_token.exp`; minted with no step-up window | @@ -1551,15 +1560,28 @@ delegated pathways rather than narrowing it, and an operator turning it off must lockout policy carrying the whole load. OIDC has no engine credential to lock out; the mTLS plane has no guessable secret at all, so no rate limit or lockout applies to it. **A genuine second factor is built for local accounts only** — TOTP (WP-14) *and* WebAuthn passkeys (WP-14b), the latter being the only -phishing-resistant factor shipped; `[auth].require_mfa` defaults on but scopes to local Administrator -accounts, and it is enforced at the **step-up boundary, not as an access gate** (see -[Multi-factor authentication](#multi-factor-authentication-totp-wp-14) and the separate 6.3.3 scoring). +phishing-resistant factor shipped; `[security].require_mfa` defaults **on** and its shipped scope is +**`every_local_account`** rather than the Administrator role, and it is enforced as an **access gate, not +only at the step-up boundary** — an MFA-pending session is refused on every authorized route. An earlier +revision of this sentence asserted the opposite on both counts and named the `[auth]` keys the loader +rejects; it also contradicted the Local row of the table above, which was right (see +[Multi-factor authentication](#multi-factor-authentication-totp-wp-14)). AD/Kerberos MFA is delegated to the directory; OIDC's is asserted by the IdP and gated on a -signature-verified claim. **The consequence, stated plainly:** every directory pathway satisfies the -engine's MFA gates without an engine-verified factor — `_complete_ad_login` mints AD, Kerberos and OIDC -sessions `mfa_verified=True` unconditionally — so a *password* on the AD pathway reaches the same PHI -surface as a passkey-backed local Administrator. OIDC is the only delegated pathway that carries any -engine-side evidence at all (the signature-verified `amr`/`acr` gate); AD and Kerberos carry none. +signature-verified claim. **The consequence, stated plainly:** the AD and Kerberos pathways satisfy the +engine's MFA gates without an engine-verified factor, so a *password* on the AD pathway reaches the same +PHI surface as a passkey-backed local Administrator. The mechanism is a **per-mechanism argument**, not a +blanket literal: `mfa_verified` is a keyword parameter of `_complete_ad_login`, and the simple-bind and +Kerberos legs pass `True` under the signed delegated-directory relaxation while the federated leg passes +`[auth].oidc_require_mfa_claim` itself — on by default, and reached only after the claim gate has already +refused any token carrying no configured `amr`/`acr`, so the grant there is engine-verified rather than +assumed. Turn that setting off and the federated session mints **un**verified, and `mfa_satisfied` +refuses it while `[security].require_mfa` is on (the default). An earlier revision of this sentence said +`_complete_ad_login` mints all three `mfa_verified=True` unconditionally; that was wrong about the +mechanism and contradicted the OIDC row of the table above. OIDC therefore remains the only delegated +pathway carrying engine-side evidence at all; AD and Kerberos carry none, and **closing that gap is the +deploying site's job, in the directory** — the engine accepts whatever the directory asserts, so the +domain's own MFA policy (Entra Conditional Access or an MFA proxy) is the only control over those two +pathways. ## Brute-force & abuse protection diff --git a/tests/test_docs_security_pathways.py b/tests/test_docs_security_pathways.py index 18bd3936..eae1b01c 100644 --- a/tests/test_docs_security_pathways.py +++ b/tests/test_docs_security_pathways.py @@ -218,7 +218,12 @@ def test_companion_table_covers_the_remaining_strength_dimensions() -> None: True, "`[auth].oidc_require_mfa_claim` defaults **on**", ), - (AuthSettings, "require_mfa", True, "`[auth].require_mfa` defaults **on**"), + # The MODEL field stays `AuthSettings.require_mfa` (the internal desugared field), but the + # OPERATOR-FACING key is `[security].require_mfa`: `("auth", "require_mfa")` is in + # `_RELOCATED_TO_SECURITY`, so the `[auth]` spelling raises at load and `serve` exits 2. The + # rendered token has to quote the key an operator can actually set, or this guard pins the + # documentation to a config that cannot start. + (AuthSettings, "require_mfa", True, "`[security].require_mfa` defaults **on**"), ( AuthSettings, "ad_session_recheck_seconds", From fa0f9631a01d39b4a9bbf9f064d880fe13a51bd6 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Mon, 17 Aug 2026 15:39:31 -0500 Subject: [PATCH 3/5] docs(security): correct the key-management scope and inventory four missing surfaces The section opened by claiming "a single, change-controlled inventory of every key, algorithm, and certificate the engine relies on". The word every was false: four shipped crypto surfaces had no row. Rows are added for all four and the opening is reworded to an at-least shape, so the next connector does not falsify it again. - the verifying SMTP hop shared by the [alerts] sink and the EMAIL and DIRECT destinations, whose one build_smtp_tls_context factory is what stands between the engine and smtplib's own unverified fallback - XML-DSig verification via signxml, which appeared once in the whole document, inside the CI blockquote, with no algorithms or trust anchor - the SFTP connector's SSH host-key trust and its RSA client key - the SMART Backend Services client assertion Storage/access for the store DEK said "environment only". That is refuted by three further shipped routes: a DPAPI-protected key file, the vault key provider (keyprovider_vault.py ships and is dispatched by name behind the [vault] extra), and cipher_provider = vault_transit, where no local key material exists at all. Only aws_kms, azure_kv, gcp_kms and pkcs11 are unbuilt; the settings comment still says all five are, which is what the old bullet was written from. A site hardening key handling from that sentence would not learn it can keep the plaintext key out of the service environment entirely. A distribution bullet is added, and stated as the true asymmetry rather than a cap: the engine needs one holder, escrow is a deliberate second holder with a stated reason, and further copies are unaccounted for by construction. Asserting "exactly one entity" would have contradicted the escrow mandate in the very next bullet. A new subsection covers the five asymmetric private keys the engine loads but does not mint. They previously had a rotation cadence and nothing else: no generation, storage, holder or destruction policy. The DEK's rules read across badly in both directions, which is why the split is stated explicitly - escrowing a signing key would copy a credential whose loss costs only a re-issue, and the DEK's discard-is-erasure property describes no other key here. The JWS row listed three of the five algorithms the enum ships, so a partner requiring SHA-384 looked unsupported. tests/test_key_usage_scope_inventory.py's curated registry classifies every parsed row; the four new labels are added to the key-material set or the guard fails on unclassified rows. --- docs/ASVS-L2-PHASE0-CHANGES.md | 97 +++++++++++++++++++++++-- tests/test_key_usage_scope_inventory.py | 4 + 2 files changed, 96 insertions(+), 5 deletions(-) diff --git a/docs/ASVS-L2-PHASE0-CHANGES.md b/docs/ASVS-L2-PHASE0-CHANGES.md index a999a1c3..9aec17f4 100644 --- a/docs/ASVS-L2-PHASE0-CHANGES.md +++ b/docs/ASVS-L2-PHASE0-CHANGES.md @@ -72,9 +72,23 @@ The WS handshake checks `Origin` the same way (§1). ## 4. Key-management & cryptographic inventory (ASVS 11.1.1 / 11.1.2) -A single, change-controlled inventory of every key, algorithm, and certificate the engine relies on. +The engine's change-controlled inventory of the keys, algorithms, and certificates it relies on. Update it whenever a crypto dependency, algorithm, or key source changes. +This section used to open by claiming a single inventory of **every** key, algorithm, and certificate +the engine relies on. That was false as written: four shipped crypto surfaces had no row in the +inventory table below — the verifying SMTP hop shared by the `[alerts]` sink and the EMAIL/DIRECT +destinations, XML-DSig verification via `signxml`, the SFTP connector's SSH host-key trust and its +client key, and the SMART Backend Services client assertion. Rows for all four are in the table now. +Read the table as covering **at least** what it lists rather than as closed, and treat a surface you +cannot find here as uninventoried rather than as absent from the engine. + +**What a deploying site still owns.** These rows describe what the engine generates, loads, and +enforces. Material a site supplies — CA bundles, partner and signer certificates, SSH host keys, PEM +private keys and their passphrases — is generated, stored, escrowed, and destroyed under that site's +own key-management policy; the table records where each input enters the engine and what it is used +for, not how it is protected before it gets there. + > **Enforced by CI (ASVS 11.1.3, WP-L3-02).** `scripts/security/crypto_inventory_check.py` is the > machine-readable companion to this section. It walks the five first-party roots (`messagefoundry/`, > `messagefoundry_webconsole/`, `harness/`, `tee/`, `scripts/` — pinned identical to @@ -104,7 +118,7 @@ Update it whenever a crypto dependency, algorithm, or key source changes. | Engine wheel attestation ([ADR 0041](adr/0041-load-path-attestation-and-change-attribution.md) D3) | SHA-256 over each **loaded** first-party `messagefoundry` module file, compared to the installed wheel's `*.dist-info/RECORD` baseline (a base64 `sha256=` manifest already in the wheel); `hashlib` in `integrity.py` | Drift recorded in the hash-chained `startup_integrity` audit row (not a secret); RECORD baseline read from site-packages metadata | Recomputed at startup + on demand; in-place-tamper tripwire (integrity, not confidentiality). Alert-only by default; `[integrity].fail_closed_on_drift` refuses to start on drift; no-op on an editable install | | ASVS corpus pin ([ADR 0156](adr/0156-asvs-scorecard-as-data-a-derived-count-verified-evidence-anchors-and-a-fail-closed-drift-gate.md)) | SHA-256 over the **OWASP ASVS 5.0.0 corpus file**, recorded in `[scorecard].corpus_sha256` and recomputed on every verifier run; `hashlib` in `scripts/asvs/scorecard.py`. **Integrity of a build input, not a security control** — no secret, no key, no message authentication, and nothing user- or PHI-derived is hashed. It exists because the corpus was originally fetched from `master` (the bleeding-edge branch, where a rolling "latest" release republishes identical filenames) and matched the tagged `v5.0.0_release` asset only by luck; the digest is now recorded and checked rather than assumed, because ASVS requirement ids are **not stable across versions** (bare `1.2.5` is *Architecture* in 4.0.3 and *Encoding and Sanitization* in 5.0.0), so a corpus that moves silently re-points every id in the scorecard | Not a secret: the digest is committed alongside the corpus it pins | Recomputed on every scorecard verification; a mismatch fails the gate and forces re-verification before any verdict is trusted | | ASVS scorecard revision identifier ([ADR 0156](adr/0156-asvs-scorecard-as-data-a-derived-count-verified-evidence-anchors-and-a-fail-closed-drift-gate.md)) | SHA-256 over the **ASVS scorecard file**, printed truncated to 16 hex characters by a `--prove-absences` run; `hashlib` in `scripts/asvs/prove_report.py`. Same class as the corpus pin above and **not a security control** for the same reasons — no secret, no key, no message authentication, nothing user- or PHI-derived. It differs only in what it covers: the record itself rather than a build input, and it is never compared against a declared value. It exists so a run states *which* revision of the record it read — two runs reporting different counts are otherwise indistinguishable from one run whose input moved underneath it | Not a secret: it is an identifier in a run log, and the scorecard it covers is private for unrelated reasons | Recomputed on every run; nothing is gated on it, so a change is information for a reader rather than a failure | -| Outbound message signing (opt-in) | Detached JWS (RFC 7515) — RS256/PS256 (RSA) or ES256 (ECDSA P-256), SHA-256; `cryptography` in `transports/signing.py` (ASVS 4.1.5, [ADR 0018](adr/0018-per-message-signatures-accepted-risk.md)) | Operator-supplied PEM **private** signing key per connection (inline via `env()` or a PEM file path; encrypted-key passphrase via `env()`); the **public** key is shared with the partner out-of-band. **Usage scope:** this private key **only** signs this connection's outbound per-message JWS — a message-**authenticity/integrity** key in transit; it is never used for at-rest encryption or session/token material, and the partner holds only the matching **public** verification half | **OFF by default**; per-connection opt-in. `kid` carried in the JWS header so key rotation / a managed provider ([ADR 0019](adr/0019-pluggable-keyprovider-hsm-kms-vault.md)) slots in without a wire change | +| Outbound message signing (opt-in) | Detached JWS (RFC 7515) — RSA `RS256`/`RS384` (PKCS#1 v1.5, deterministic) or `PS256` (PSS), or ECDSA `ES256` (P-256) / `ES384` (P-384); SHA-256 except `RS384`/`ES384`, which are SHA-384. This row previously listed only the three SHA-256 algorithms, but the shipped `SignatureAlgorithm` enum has five members and `OutboundSigning.algorithm` accepts any of them, so the SHA-384 pair is available for per-message signing too; `cryptography` in `transports/signing.py` (ASVS 4.1.5, [ADR 0018](adr/0018-per-message-signatures-accepted-risk.md)) | Operator-supplied PEM **private** signing key per connection (inline via `env()` or a PEM file path; encrypted-key passphrase via `env()`); the **public** key is shared with the partner out-of-band. **Usage scope:** this private key **only** signs this connection's outbound per-message JWS — a message-**authenticity/integrity** key in transit; it is never used for at-rest encryption or session/token material, and the partner holds only the matching **public** verification half | **OFF by default**; per-connection opt-in. `kid` carried in the JWS header so key rotation / a managed provider ([ADR 0019](adr/0019-pluggable-keyprovider-hsm-kms-vault.md)) slots in without a wire change | | DIRECT S/MIME (opt-in, [ADR 0085](adr/0085-direct-hisp-smime-connector.md)) | CMS **sign-then-encrypt** in `transports/direct.py` (core `cryptography` `serialization.pkcs7`): PKCS#7 signature over the body with a **SHA-256** digest, the public-key signature algorithm (RSA / ECDSA) following the loaded signing key type (not pinned to RSA), then a PKCS#7 **envelope** to the partner's recipient cert. The envelope content-encryption cipher is the **`cryptography` pkcs7 library default** — no algorithm is pinned in code | Sender **signing cert** + PEM **private key** (optional `signing_key_password`) and the per-partner **`recipient_cert`**, all operator-supplied files; the recipient cert is trust-verified at construction against an operator `trust_anchor` (one-level direct-issuance check); key/cert mismatch refused. **Usage scope:** the sender signing key signs the CMS body and the partner's `recipient_cert` encrypts the CMS envelope — this material protects the **confidentiality + authenticity of a DIRECT message to one partner in transit**; it is not an at-rest store key and encrypts nothing in the store | **OFF by default** — only when a DIRECT Connection is configured, and its HISP relay host is gated by the **opt-in** `[egress].allowed_direct` allow-list (empty by default = unrestricted; an unlisted host is refused only once the list is populated, or outright when `[security].block_unlisted_outbound` is set). Signing key + recipient certs rotate on the schedule below | | OIDC IdP JWKS verification keys (opt-in, [ADR 0142](adr/0142-federated-sso-oidc-authorization-code-pkce-relying-party-hybrid-ad-backed.md)) | **Public** verifying keys fetched from the IdP JWKS: **RS256/PS256** (RSA, ≥ 2048-bit floor) and **ES256/ES384** (EC P-256/P-384) — rebuilt from each JWK by `cryptography` in [`auth/oidc/jwks.py`](../messagefoundry/auth/oidc/jwks.py); the closed `SignatureAlgorithm` enum forecloses `alg:none` and RS256→HS256 confusion. Bounded, TTL-cached (`DEFAULT_JWKS_TTL_SECONDS`), a 512 KiB body cap, a global min-refetch floor (fetch-amplification bound), and a hard refusal of a duplicate `kid`; a key below the floor is skipped/refused, never merely warned. **Usage scope:** these are **public**, non-secret keys used **only** to verify the IdP's id-token signature at console login — they encrypt nothing and can protect no data; the engine holds no private half. | Fetched from the IdP JWKS URI over the CA-pinned no-redirect opener (row below); held process-local in `JwksCache`, never persisted, never logged | Refetched per TTL / on an unknown `kid` within the amplification bound; rolls when the IdP rotates its signing keys | | OIDC IdP TLS trust anchor (`[auth].oidc_tls_ca_cert_file`, opt-in, [ADR 0142](adr/0142-federated-sso-oidc-authorization-code-pkce-relying-party-hybrid-ad-backed.md)) | Pins the CA that must anchor the IdP's TLS server cert on **both** federated-SSO legs (JWKS fetch + token endpoint) — the hardened, no-redirect `ssl` opener in [`auth/oidc_http.py`](../messagefoundry/auth/oidc_http.py) (mirrors `ad_tls_ca_cert_file`); unset ⇒ the OS trust store via `truststore`. No insecure/`verify=False` escape exists — the IdP hop carries an authentication assertion. **Usage scope:** a **trust anchor**, not a key the engine holds — it authenticates the IdP endpoint's TLS identity only; it signs and encrypts nothing and protects no at-rest data. | Operator-supplied CA PEM path (`[auth].oidc_tls_ca_cert_file`) or the OS trust store | Managed by the operator / OS trust store; rotate on IdP CA change | @@ -115,15 +129,41 @@ Update it whenever a crypto dependency, algorithm, or key source changes. | Cert tooling — `.pfx` import / read-only inventory / self-signed dev cert (BACKLOG #71/#72) | `cryptography` in [`pki.py`](../messagefoundry/pki.py) — the single PKI call site for the `cert` CLI group: PKCS#12/.pfx import (`pkcs12.load_key_and_certificates`) writes the leaf cert + private key + CA chain to the PEM files the TLS loaders already read; a **read-only** inventory reads only **public** cert facts (subject/issuer/notAfter/SAN/days) via `x509`; `make_self_signed` mints an **EC P-256 / SHA-256** self-signed cert for **non-prod** bring-up. `pipeline/cert_expiry.py` shares this module's `read_cert_facts` (so it no longer imports `cryptography` itself). **Usage scope:** an operator CLI utility — it imports/serializes/inspects operator-supplied cert material and mints throwaway dev certs; it holds no long-lived engine key, signs no message, and encrypts nothing at rest. The imported/minted **private-key** PEM is written `O_EXCL` + `0o600` + the `_secure_file` DACL; the `.pfx` passphrase is env-only (`MEFOR_PFX_PASSWORD`), never a CLI arg, never logged/echoed/put in an exception | Operator-supplied `.pfx` bundle → cert/key/CA PEM files on disk (`--out-dir`) | Managed by the operator / PKI; self-signed dev certs are disposable (default 365-day validity) | | Engine/console seam identity (BACKLOG #1220) | SHA-256 over the **discovered** engine/console contract surface, truncated to 16 hex characters, published as `ENGINE_UI_SEAM` in [`api/_ui_seam.py`](../messagefoundry/api/_ui_seam.py) and derived by `hashlib` in [`scripts/webconsole_seam_snapshot.py`](../scripts/webconsole_seam_snapshot.py). **A change detector, not a security control** — no secret, no key, no message authentication, and nothing user- or PHI-derived is hashed; the input is a serialization of public type signatures, field names, enum members and `Literal` values. It replaced a hand-picked incrementing integer, which two unlanded branches had both claimed for two different contract changes while the golden snapshot auto-merged clean under one value. What it needs is accidental-collision avoidance across the contract surfaces this project will ever produce: at 64 bits the birthday bound is 2.7e-12 for 10,000 distinct surfaces, about 500x the ~20 seam moves to date. Preimage resistance buys nothing — anyone able to craft a colliding surface already has commit access to the file holding the constant. SHA-256 rather than BLAKE2 or a non-approved digest only because the engine renders a `fips_mode` attestation and a non-approved hash in the shipped surface invites a FIPS question for no gain | Not a secret: the digest is committed in source and mirrored in the console's `SUPPORTED_ENGINE_SEAMS` | Recomputed by `scripts/webconsole_seam_snapshot.py --write` whenever the contract changes; a stale value fails `tests/test_webconsole_seam_snapshot.py` | | DAST scan-target credential ([ADR 0155](adr/0155-dast-dynamic-security-testing-of-the-running-engine.md)) | Throwaway per-run password — URL-safe CSPRNG (`secrets.token_urlsafe(24)`, ~32 characters) in [`scripts/security/dast_target.py`](../scripts/security/dast_target.py) — authenticating the two ephemeral identities (`dast-admin`, `dast-viewer`) the authenticated authorization sweep provisions. It is hashed by the same argon2id path as any local account. Generated rather than checked in **because a constant is strictly weaker and goes stale**; it is never logged, never written to the receipt artifact, and cannot reach a real deployment (the sweep creates its own store). Not key material: it protects nothing at rest or in transit and unlocks no key | Generated in-process per run; the optional `MEFOR_DAST_SCAN_PASSWORD` variable is a by-hand-repro escape hatch only. Stored only as the argon2id hash inside the scan's own SQLite store, which the runner creates **empty in a temporary directory** | Lives for exactly one scan — the temporary store is deleted with the run, so there is nothing to rotate | +| Outbound SMTP transport TLS (the `[alerts]` sink, the EMAIL destination, the DIRECT HISP relay) | One verifying `ssl` context per hop, built by `build_smtp_tls_context` in [`config/tls_policy.py`](../messagefoundry/config/tls_policy.py) and passed explicitly to `starttls()`: chain + hostname verification, a TLS 1.2 floor, the approved ECDHE groups, forward-secret cipher suites, and strict RFC 5280 verify flags. All three SMTP call sites route through that one factory ([`pipeline/alert_sinks.py`](../messagefoundry/pipeline/alert_sinks.py), [`transports/email.py`](../messagefoundry/transports/email.py), [`transports/direct.py`](../messagefoundry/transports/direct.py)), so one policy decides every SMTP hop. Passing **no** context is the failure this row exists to foreclose — `smtplib`'s own fallback is `ssl._create_unverified_context` (`CERT_NONE`, `check_hostname=False`), which encrypts the hop without authenticating the relay. **Usage scope:** a **trust anchor**, not a key the engine holds — it authenticates the relay's TLS identity on the alert / security-notification / DIRECT-relay hops only; it signs nothing, encrypts nothing at rest, and has no private half. | The connection's own CA PEM (`[alerts].email_tls_ca_file`, a connector `ca_file`), else the instance `[tls].internal_ca_file`, else the OS trust store | Managed by the operator / OS trust store; rotate on relay CA change. Turning verification off is a **loosening**, and the callers gate it differently: the `[alerts]` sink needs `[security].allow_unverified_alert_smtp_tls` at the serve gate, while the EMAIL/DIRECT connectors read the clamped `MEFOR_ALLOW_INSECURE_TLS` escape | +| Inbound XML-DSig verification (opt-in, the `[xml]` extra) | Enveloped XML-DSig signature + digest verification through `signxml` (which delegates to `cryptography`) in [`parsing/xml/signature.py`](../messagefoundry/parsing/xml/signature.py) — `signxml.XMLVerifier().verify()`, called on demand by a Handler against a signed XML/SOAP body that has already been through the hardened lxml parser (XXE/DTD lockdown). **A trust anchor is required and its absence is refused:** `verify()` raises `ValueError` unless the caller pins the expected signer certificate (`x509_cert`) or names a partner CA (`ca_pem_file`), because signxml's own default would accept any signature chaining to the host's system CA store. A failed verification is returned as data (`XmlSignatureResult`) with a PHI-safe reason category, never the document. **Usage scope:** verification material only — the engine holds **no** private half on this path, signs nothing, and encrypts nothing; it establishes the origin and integrity of an inbound document and nothing else. | Operator-supplied signer certificate or partner CA PEM, passed per call by the Handler; the engine stores neither | Rotate at the partner. `signxml` ships behind the optional `[xml]` extra, so a base install has no XML-DSig path at all. **Operator-owned:** choosing the anchor is the deploying site's decision — a Handler that passes a broadly-trusted CA verifies far more signers than it intends, and nothing in the engine can detect that | +| SFTP transport — SSH host-key trust and client key (opt-in, the `[sftp]` extra) | `paramiko` in [`transports/remotefile.py`](../messagefoundry/transports/remotefile.py) (`_SftpClient`). **Server authentication:** the system `known_hosts` plus an optional per-connection `known_hosts` file under paramiko's `RejectPolicy`, so an unknown host key is refused at connect. `AutoAddPolicy` is reachable only through the `MEFOR_ALLOW_INSECURE_TLS` escape read via the [ADR 0092](adr/0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md) clamp — inert wherever the construction posture is an enforcing PHI hop — and it logs a warning where it does apply. **Client authentication:** an operator-supplied **RSA** private key (`private_key`, passphrase `key_password`) loaded via `paramiko.RSAKey.from_private_key`, or a password; RSA is the only key type the loader builds today, so an Ed25519 or ECDSA key is not usable here. `allow_agent=False` and `look_for_keys=False`, so no ambient agent key or `~/.ssh` key is ever picked up. **Usage scope:** transport authentication for one SFTP endpoint — the client key proves *this engine* to that server; it encrypts nothing at rest and signs no message. | Host keys from the OS `known_hosts` and the connection's `known_hosts` file; the client key is `env()`-sourced PEM (`private_key`) with `key_password`, both `/metadata`-redacted | Rotate at the partner; re-pin the host key on a server rebuild. `paramiko` behind the optional `[sftp]` extra. **Operator-owned:** the engine can refuse an unknown host key but cannot tell a correct pinned key from a wrong one — populating `known_hosts` is the deploying site's job | +| SMART Backend Services client assertion (opt-in, [ADR 0024](adr/0024-smart-backend-services-token-provider.md)) | A signed compact JWT minted by `CompactJwtSigner` in [`transports/signing.py`](../messagefoundry/transports/signing.py) and exchanged for a short-lived bearer at the authorization server's token endpoint ([`transports/smart.py`](../messagefoundry/transports/smart.py)). `smart_algorithm` defaults to **RS384**; `ES384` is the other algorithm SMART Backend Services requires, and the setting accepts any of the five JWS algorithms the ADR 0018 signer supports, so pinning a SHA-256 member is possible and is a deliberate departure from the SMART profile. Signing runs over core `cryptography`, no new dependency. The assertion lives 240 seconds (under SMART's 5-minute ceiling), its `aud` is bound to the operator-pinned `token_url` so it is not replayable at another authorization server, that endpoint rides the same `[egress].allowed_http` allow-list as the FHIR data host, and a cleartext `http` token endpoint is refused by the instance security posture unless the hop is attested or cleartext is explicitly accepted with a reason. **Usage scope:** this private key **only** signs this connection's client assertion — it authenticates *the engine* to one FHIR authorization server. It is not an at-rest, session, or message-signing key, decrypts nothing, and the FHIR server holds only the matching public half. | Operator-supplied PEM private key `smart_private_key` via `env()` (passphrase `smart_private_key_password`), `/metadata`-redacted; the minted assertion and the access token are never logged or persisted | **OFF by default** — present only when a SMART-authorized FHIR outbound is configured. Cadence in the rotation schedule below: register the new public key at the FHIR server first, then replace the PEM | | Engine-shard lane ownership ([ADR 0073](adr/0073-ownership-scoped-recovery-single-consumer-lanes.md)) | Rendezvous (HRW) hash — SHA-256 (`hashlib` in `pipeline/sharding.py`) over `destination + shard id`, picking each outbound lane's single delivering shard. A stable, process-independent hash is required (the salted builtin `hash()` would let two shards disagree on an owner); deterministic placement, **not** a security control | No key material — pure function of config names | Recomputed per process from the loaded config; changes only with the shard universe (coordinated fleet restart) | ### Store-key management policy (NIST SP 800-57 alignment) - **Generation** — mint with `messagefoundry gen-key` (32 bytes from `os.urandom`); supply via `MEFOR_STORE_ENCRYPTION_KEY`, **never** the TOML file. -- **Storage / access** — environment only; the process account is the trust boundary. Restrict the - data-volume and service account (see [SERVICE.md](SERVICE.md)); volume encryption - (BitLocker/LUKS) is the required at-rest layer for the columns outside the cipher. +- **Storage / access** — the process account is the trust boundary. This bullet used to say + "environment only"; that is **no longer true**, and an operator who read it would miss shipped + options. There are **at least four** routes today: the `MEFOR_STORE_ENCRYPTION_KEY` environment + value (the cross-platform default); a Windows DPAPI-protected file named by + `[store].encryption_key_file` and written by `messagefoundry protect-key` — a *path*, not a secret, + so it may live in the TOML file, and the environment value wins when both are set; + `[store].key_provider = vault`, which envelope-decrypts a **wrapped** DEK against a non-extractable + key inside HashiCorp Vault/OpenBao Transit (behind the optional `[vault]` extra, so a base install + pulls no SDK); or `[store].cipher_provider = vault_transit`, where there is no local key material at + all, because the bulk encrypt/decrypt happens inside Transit and `encryption_key` / `key_provider` go + unused. `[store].key_provider` also accepts `auto` (the default: environment then DPAPI) and the + pinned `env` / `dpapi`. The remaining external provider names — `aws_kms`, `azure_kv`, `gcp_kms`, + `pkcs11` — are designed but **not built**, and selecting one fails closed at store open rather than + degrading to the plaintext cipher. Restrict the data-volume and service account (see + [SERVICE.md](SERVICE.md)); volume encryption (BitLocker/LUKS) is the required at-rest layer for the + columns outside the cipher. +- **Distribution / holders** — the engine needs exactly **one** holder: the process account, by + whichever route above the site chose. Where the engine reports on the key it reports a fingerprint + and not the key — the security-posture view carries only the active key's one-way `key_id` + (`CipherInfo` in [`store/crypto.py`](../messagefoundry/store/crypto.py)). The escrow requirement in + the next bullet is therefore a **deliberate second holder**, and the only copy outside the running + process this project asks for: escrow once, into the narrowest custody the site's recovery procedure + needs. Every further copy — a value pasted into a ticket, a backup of the config directory, a secret + manager replicated across regions — is a holder the engine can neither see nor revoke, and counting + them is the deploying site's responsibility, not something the engine can do for it. - **Rotation / retirement** — **built (WP-5, ASVS 11.2.2).** Each ciphertext is self-identifying via its `key_id` (SHA-256 fingerprint), so multiple keys coexist: set the new key as `MEFOR_STORE_ENCRYPTION_KEY`, move the prior one to `MEFOR_STORE_ENCRYPTION_KEYS_RETIRED` @@ -137,6 +177,53 @@ Update it whenever a crypto dependency, algorithm, or key source changes. connection labels) — never a body — which relies on volume encryption (accepted residual — see [PHI.md §3](PHI.md#3-encryption-at-rest)). +### Key management for the other private keys the engine loads + +The bullets above govern exactly **one** key — the store DEK — and they do not generalize. The +schedule below has always carried the other keys, but it answers a single question about them (how +often to rotate, and what to replace) and is silent on the rest: who mints them, where they may live, +how many holders are acceptable, and what retiring one actually means. That is this subsection. + +**What the engine mints.** Of the long-lived asymmetric and at-rest key material on this page, the +engine mints exactly two things; every other key below is minted by the deploying site or by a +counterparty, and the engine only loads it. (Short-lived CSPRNG secrets the engine also generates — +session tokens, WebAuthn ceremony challenges, the throwaway scan credential — have their own rows in +the inventory table above.) The two are the store DEK (`messagefoundry gen-key` — 32 bytes from +`os.urandom`, base64-encoded, printed to stdout and not persisted by the command; `generate_key` in +[`store/crypto.py`](../messagefoundry/store/crypto.py)), and a **non-production** self-signed EC P-256 +certificate and key (`messagefoundry cert self-signed`, `make_self_signed` in +[`pki.py`](../messagefoundry/pki.py) — the key PEM is unencrypted PKCS#8, written `O_EXCL` with mode +`0o600` and a tightened Windows DACL, refusing to overwrite an existing file; a self-signed +certificate has no chain of trust and must never front production PHI). `messagefoundry cert import` +unpacks an operator's PKCS#12 bundle into the PEM files the TLS loaders read — it relocates key +material, it creates none. + +| Key | How it reaches the engine | Who holds it | What losing it costs | +|---|---|---|---| +| **Per-message JWS signing key** — the `with_signing` parameters `private_key` / `private_key_password`, which reach a REST/SOAP outbound as the flat settings `sign_private_key` / `sign_private_key_password` ([ADR 0018](adr/0018-per-message-signatures-accepted-risk.md)) | inline PEM through `env()`, **or** a path to a PEM file the OS protects; read once when the connector is constructed (`_load_private_key` in [`transports/signing.py`](../messagefoundry/transports/signing.py)) | the engine process, plus wherever the operator keeps the PEM. Only the public-verifiable signature leaves the box; the receiver verifies against the matching **public** key, exchanged out of band per partner contract | re-issuing the key and republishing the public half to each partner. No stored message becomes unreadable | +| **SMART Backend Services client-assertion key** — `smart_private_key` (+ `smart_private_key_password`) ([ADR 0024](adr/0024-smart-backend-services-token-provider.md)) | the same two forms, resolved through `env()` and loaded by the same signer | the engine process and the operator's secret store; only the signature and the registered `kid` leave the box | re-registering the replacement key with the FHIR server | +| **DIRECT S/MIME sender signing key** — `signing_key` (+ `signing_key_password`) ([ADR 0085](adr/0085-direct-hisp-smime-connector.md)) | a **file path only** — the engine reads the PEM/DER bytes from disk at construction and checks the key against `signing_cert`, refusing a pair whose public keys differ. `signing_key` names a path, so it is deliberately **not** a secret setting: it is the file, not the setting, that has to be protected | the operator's file system, read by the process account; the recipient holds the sender's certificate | replacing the key and certificate and re-exchanging with partners. This connector **sends** only, so nothing already received depends on it | +| **SFTP client key** — `private_key` (+ `key_password`) on a remote-file connection | **inline PEM only** — the value is parsed as key text and never opened as a path — and only an **RSA** key loads today, so an Ed25519 or ECDSA client key is not usable with this connector | the operator's secret store and the engine process; the peer holds the matching public key | enrolling a new public key with the SFTP peer | +| **API TLS server key** — `[api].tls_key_file` (+ `MEFOR_API_TLS_KEY_PASSWORD`) | a PEM file path handed to the TLS stack when the listener's context is built; the key may instead be embedded in the certificate PEM, in which case `tls_key_file` is omitted. A bad PEM, or a wrong or missing passphrase, raises at that construction point — **before** the socket opens — rather than degrading to plaintext | the operator's file system, read by the process account | re-issuing from the site's CA; the listener will not start until a usable key and certificate are present | + +**Distribution.** For all five the engine needs **one** holder: the process account, reading an +`env()`-resolved value or a file. It escrows none of them, and none should be escrowed on the DEK's +reasoning — the DEK is escrowed because losing it strands rows nothing else can recover, whereas +losing any of these five costs a re-issue and a conversation with a counterparty, never data. The +right holder count for them is the lowest the site can operate with. The connection settings that +carry key text or a passphrase are `/metadata`-redacted (`_SECRET_SETTING_KEYS` and +`_is_secret_setting` in [`config/wiring.py`](../messagefoundry/config/wiring.py)), so a console +operator cannot read a key value back out of a running engine; the **path**-valued connection setting +`signing_key` is served as the path it is, which is exactly why the file behind it needs file-system +permissions of its own. + +**Destruction and retirement belong to the deploying site.** The engine ships no command that +destroys, revokes or expires any of these five. It stops using one when the connection's setting +stops naming it, and the PEM keeps working anywhere it was copied — so retiring a key means revoking +or de-registering it at the counterparty and deleting the operator-side copies. Only the store DEK +has the property the Destruction bullet above describes, where discarding the key is itself the +erasure. + ### Rotation schedule (ASVS 13.1.4 / 13.3.4) A rotation cadence per critical secret, justified against the threat model + HIPAA. These are diff --git a/tests/test_key_usage_scope_inventory.py b/tests/test_key_usage_scope_inventory.py index e4cca75f..9aa46ffe 100644 --- a/tests/test_key_usage_scope_inventory.py +++ b/tests/test_key_usage_scope_inventory.py @@ -50,6 +50,10 @@ "OIDC IdP JWKS verification keys", "OIDC IdP TLS trust anchor", "Cert tooling", + "Outbound SMTP transport TLS", + "Inbound XML-DSig verification", + "SFTP transport", + "SMART Backend Services client assertion", } ) From 53f4b5238cc720659c06172f86f91702b640253e Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Mon, 17 Aug 2026 15:39:49 -0500 Subject: [PATCH 4/5] docs(connections): correct the file-surface inventory and the DICOM size ceilings The ASVS 5.1.1 policy block opened with a closed count - "MessageFoundry's file surface has three parts" - that omitted the DICOM C-STORE SCP, a fourth surface on which a remote modality pushes whole objects into the engine. A reader auditing the file-receiving surface from that sentence would have missed it, and would have had no signal that the drop-directory policy below does not govern it. Reworded to an at-least shape with the SCP named and its controls pointed at the DICOM section. The decompression bullet claimed that with no decompress= set "there is no unpacked-size surface". False for a shipped payload type: a Deflated Explicit VR LE DICOM object carries its own DEFLATE stream, the drop's content sniff accepts it on the DICM magic alone, and the inflate is bounded elsewhere - at 16 MiB with no per-connection knob when a Router or Handler parses it, and at max_object_bytes when an outbound SCU forwards it. Neither ceiling appeared anywhere in operator documentation. Measured over all tracked paths under docs/ with per-file grep -c: guard_part10_deflate, MAX_INFLATED and "Deflated Explicit VR" each matched exactly one file, the ledger, against a positive control of max_object_bytes matching five files in the same run. The inbound max_object_bytes row now records the second duty that setting carries and the counter-intuitive consequence of disabling it: 0/None does not widen the limit, it removes the object-size check and tightens the inflate ceiling to the 16 MiB codec default the guard falls back to. The bullet stops short of claiming an exposure: max_file_bytes still caps the compressed bytes at ingest where that cap is set, and whether a Handler parses the object is site-specific, so the text says where the bound lives. --- docs/CONNECTIONS.md | 33 ++++++++++++++++++++++++++------- 1 file changed, 26 insertions(+), 7 deletions(-) diff --git a/docs/CONNECTIONS.md b/docs/CONNECTIONS.md index 513d360d..8043d4a1 100644 --- a/docs/CONNECTIONS.md +++ b/docs/CONNECTIONS.md @@ -715,14 +715,23 @@ at the source. #### File handling & quarantine policy (ASVS 5.1.1) -MessageFoundry's file surface has three parts: the **directory sources** (the local `File(...)` and -remote `Sftp(...)`/`Ftp(...)` connectors) that ingest drop-directory files into the pipeline; the +MessageFoundry's file surface has **at least four** parts. This list is maintained by hand, so read it +as the current inventory and not as a closed set. The **directory sources** (the local `File(...)` and +remote `Sftp(...)`/`Ftp(...)` connectors) ingest drop-directory files into the pipeline; the **opt-in HTTP uploaded-logs upload** (POST `/uploads` + the web-console delegate POST `/ui/uploaded-logs/upload`, [ADR 0134](adr/0134-offline-uploaded-logs-viewer-connection-decoupled-upload-browse-resend-deletion-phi-at-rest-posture-stdlib-multipart.md)) -for operator diagnostic logs; and the **attachment download** route (GET +carries operator diagnostic logs; the **attachment download** route (GET `/messages/{message_id}/attachments/{attachment_id}`, [ADR 0105](adr/0105-streaming-very-large-hl7-attachments-detach-the-opaque-document-from-the-transformable-skeleton.md)) -that serves a detached document back out. The **directory source's** handling of an untrusted drop -directory is fixed policy (the HTTP uploaded-logs surface has its own policy block below): +serves a detached document back out; and the **DICOM C-STORE SCP** (an inbound `DICOM(...)`, +[ADR 0025](adr/0025-dicom-codec-store-connectors.md)) receives whole objects **pushed by a remote +modality** over DIMSE. An earlier revision of this sentence said "three parts" and omitted the SCP; +that enumeration was wrong — the SCP is a receiver of remote-pushed content on the same footing as the +two HTTP routes. None of the drop-directory policy below applies to it: its size ceilings, peer +controls and transport security are connector settings documented under +[DICOM](#dicom--dicom-inbound-c-store-scp--outbound-c-store-scuc-echo-and-dicomweb-stow-rs-adr-0025), +and a deploying site must set them there rather than assume this block covers them. The **directory +source's** handling of an untrusted drop directory is fixed policy (the HTTP uploaded-logs surface has +its own policy block below): - **Permitted type — the inbound's declared `content_type` (default `hl7v2`).** Files are selected by the `pattern` glob (default `*.hl7`), and every candidate is **content-sniffed against that declared @@ -739,7 +748,17 @@ directory is fixed policy (the HTTP uploaded-logs surface has its own policy blo - **Maximum size.** `max_file_bytes` (default **16 MiB**, matching the MLLP frame cap). An oversize file is rejected by a `stat()` **before** it is read into memory (OOM / DoS guard); `None`/`0` disables it. - **Decompression is off by default; opt-in single-stream gzip is bomb-guarded** (ADR 0123). With no - `decompress=` set the connector reads raw bytes only and there is no unpacked-size surface. When + `decompress=` set the connector performs no decompression itself, so it materialises nothing beyond + `max_file_bytes` where that cap is set. An earlier revision went further and said there is "no + unpacked-size surface"; that was wrong — a file's *payload* can carry its own compressed stream. The + shipped case is a **Deflated Explicit VR LE** DICOM object, which the drop's content sniff accepts on + the `DICM` magic alone and which therefore reaches the pipeline with its inflated size unexamined. + That inflate is bounded where the object is unpacked rather than at ingest: at **16 MiB**, with no + per-connection knob, when a Router or Handler parses it (`guard_part10_deflate` in + `parsing/dicom/_inflate.py`, called from `DicomPeek.parse` and `DicomDataset.parse`), and at + `max_object_bytes` when an outbound C-STORE SCU forwards it. A site dropping DICOM into a watch + directory should size those two ceilings deliberately rather than read this bullet as saying no + unpacking happens. When `decompress="gzip"` is enabled it gunzips each drop **before** the content sniff, the AV scan, and the batch split (so all three see the real bytes), and `max_decompressed_bytes` (default 64 MiB) caps the *decompressed* size — a decompression-bomb guard the compressed-only `max_file_bytes` cap cannot @@ -1602,7 +1621,7 @@ MWL, Query/Retrieve (C-FIND/C-MOVE/C-GET), and pixel-data handling. | `presentation_contexts` | `None` → SR + common image storage + Verification | the SOP classes the SCP negotiates (transfer syntaxes default to the standard set) | | `calling_ae_allowlist` | `None` → any (subject to the IP gate) | only these calling AE titles may associate (fail-closed when set) | | `require_called_ae_title` | `True` | a peer must address this engine's `ae_title` as the called AE | -| `max_object_bytes` | `134217728` (128 MiB) | reject a single C-STORE object larger than this **before** the durable commit (OOM/DoS guard) | +| `max_object_bytes` | `134217728` (128 MiB) | reject a single C-STORE object larger than this **before** the durable commit (OOM/DoS guard). It is **also** the ceiling for the pre-decode **inflate** of a *Deflated Explicit VR LE* object: the SCP bound-inflates the raw received Data Set before pydicom touches it, so an over-cap deflate bomb is a DIMSE failure and is never decoded or committed. Note that `0`/`None` does not simply widen this — it removes the object-size check entirely **and tightens** the inflate ceiling to the codec default of **16 MiB**, which is what the guard falls back to when no object cap is configured | | `max_associations` | `10` | cap on concurrent inbound associations (connection-flood guard) | | `max_pdu_size` | `16384` | cap one PDU's bytes (`0` = unbounded); DoS guard | | `timeout_seconds` | `30.0` | ACSE/DIMSE/network timeout | From a19bcaf06cd60e9a869bcd36b6ebe53a4db24698 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Mon, 17 Aug 2026 15:40:18 -0500 Subject: [PATCH 5/5] docs(retention): correct the PHI retention startup gate on both enforcement dials Three documents stated the refuse / auto-bound split backwards, and this is the one correction in the set where a site acting on the old text loses PHI. They said a PHI instance under [security].enforcement = enforce refuses to start (exit 2) when a PHI-body window is unbounded, and that the 30-day auto-bound applied only to a non-enforcing instance. The shipped gate in __main__.py does the opposite for an UNSET window: the defaulting loop is guarded by `if not settings.retention.allow_unbounded_phi:` alone, with no enforcement branch, so an unset window is defaulted to 30 days and the instance starts on both dials. tests/test_cli.py already pins this at --env prod on the shipped enforce dial, asserting rc == 0 and a 30-day window. The practical harm is specific: a site could deliberately leave a window unset as a hold, expecting the boot to stop until someone chose a number, and would instead get a started instance that begins purging PHI bodies at 30 days. The old sentences are named as retired rather than silently swapped, because a reader who acted on them needs to see that they changed. What survives is the fail-closed path for an EXPLICIT 0, which is not auto-bounded and still refuses under enforce (warns under warn). The closing claim that a PHI instance "cannot run with PHI-body retention off without a loud, audited opt-out" is narrowed accordingly - it holds under enforce, but under warn an explicitly-zeroed window warns and starts with no opt-out. Related corrections in the same gate: - the set of auto-bounded windows is three, not the two the prose implied, and reference_snapshot_days was missing. The windows that carry no auto-bound are described as an open set with the two clearest cases and their reason, rather than a count: six of the nine classified windows carry auto_bound_days=None, each for its own recorded reason, so naming two as "the" exclusions would have been a fresh false enumeration. - the [retention] block is no longer one of the blocks a stock PHI instance must configure to boot, so the worked example's gate count and its dead_letter_days comment (REQUIRED, "its own exit 2") are corrected to RECOMMENDED with the reason to set it anyway. - the loosening register credited enforcement = warn with the auto-bound. It is dial-independent; what warn actually changes for retention is that an explicit 0 warns instead of refusing. Both files also now say plainly that 30 days is the engine's floor against an accidentally unbounded window and not the site's retention policy, since an auto-bounded window is a number nobody decided. --- docs/CONFIGURATION.md | 41 ++++++++++++++++++++++++++------------ docs/PHI.md | 37 ++++++++++++++++++++++++++++------ docs/SECURITY-LOOSENING.md | 13 +++++++----- 3 files changed, 67 insertions(+), 24 deletions(-) diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index e747e9b8..fbde2455 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -716,11 +716,19 @@ disposition, and the audit trail stay intact — the Mirth Data-Pruner pattern); `messages` row and never touches a body still in flight. The *row* survives; its PHI *columns* do not — `messages.metadata` is nulled in the same statement as the body (ASVS 14.2.7). The raw `[retention]` fields still default to `0`/`""` = keep/off, **but `serve` applies a posture gate on top of them, so retention is *not* -opt-in on a PHI instance**: under `[security].enforcement = enforce` (the default) an unbounded -`[security].delete_message_bodies_after_days` or `[retention].dead_letter_days` **refuses to start -(exit 2)**; on a non-enforcing PHI instance each *unset* window is auto-bounded to **30 days**. All -three built-in environment names (`dev`, `staging`, `prod`) derive PHI. The audited opt-out is -`[security].allow_keeping_phi_indefinitely = true`. See [PHI.md §8](PHI.md#8-retention--purge). +opt-in on a PHI instance**: each *unset* window that carries an auto-bound — +`[security].delete_message_bodies_after_days`, `[retention].dead_letter_days` and +`[retention].reference_snapshot_days` — is **defaulted to 30 days** at startup, under **both** +`[security].enforcement` dials, and the defaulted settings are named on stderr. A window set +**explicitly to `0`** is not defaulted: that **refuses to start (exit 2)** under `enforce`, and warns +under `warn`. This paragraph used to state the opposite split — refusal under `enforce`, auto-bound +only on a non-enforcing instance — which the shipped gate in +[`__main__.py`](../messagefoundry/__main__.py) refutes; an *unset* window has not refused since the +auto-bound moved to both dials. All three built-in environment names (`dev`, `staging`, `prod`) +derive PHI. The audited opt-out is `[security].allow_keeping_phi_indefinitely = true`, which +suppresses the auto-bound as well as the refusal. **Thirty days is the engine's floor against an +accidentally unbounded window, not your retention policy — set each window to the number your site +actually requires.** See [PHI.md §8](PHI.md#8-retention--purge). | Key | Type | Default | Notes | |---|---|---|---| | `messages_days` | | | **→ moved to `[security].delete_message_bodies_after_days`** (ADR 0118) — set it there; no longer accepted in `[retention]`. | @@ -1518,7 +1526,7 @@ and a PHI weakening under **strict enforcement** (`enforcement = enforce`, the d | `sign_out_after_idle_minutes` | int | `30` | session idle timeout | | `max_session_hours` | int | `12` | session absolute lifetime | | `block_unlisted_outbound` | bool | `true` | deny-by-default egress — only allow-listed destinations send. **Leaving it unset does not apply `true`** — the internal flag stays `false` and the `[egress]` startup gate decides; see the note under this table | -| `delete_message_bodies_after_days` | int | `30` | bounded PHI-body retention; `0` = keep indefinitely (audited). **Leaving it unset does not apply 30** — the internal window stays `0` and the `[retention]` startup gate decides (refuse under `enforce`, auto-bound to 30 under `warn`); see the note under this table | +| `delete_message_bodies_after_days` | int | `30` | bounded PHI-body retention; `0` = keep indefinitely (audited). **Leaving it unset does not apply 30 through the desugar** — the internal window stays `0`, and the `[retention]` startup gate then defaults it to 30 days on a PHI instance under **either** enforcement dial. This row used to say the gate refuses under `enforce` and auto-bounds only under `warn`; it does not — only an **explicit** `0` reaches the refusal. See the note under this table | | `allow_keeping_phi_indefinitely` | bool | `false` | audited escape: unbounded PHI retention | | `audit_all_authorization_decisions` | bool | `false` | ePHI access is **always** audited regardless of this switch; this adds full *authorization-decision* tracing on top (off by default — forcing it on risks flooding the audit log). "Always audited" is about **coverage**, not about how hard those rows are to alter afterwards: the audit chain is only cryptographically tamper-*evident* on a **keyed** store, and its verify does not catch a truncated tail — see [`[integrity]`](#integrity) | | `handles_real_patient_data` | bool | *derived* | the master data-class lever (was `[ai].data_class = "phi"`). Unset ⇒ derived from the environment name — **all three built-in names (`dev`/`staging`/`prod`) now derive PHI** ([ADR 0148](adr/0148-phi-default-posture-and-an-explicit-security-enforcement-level.md) GIVEN 1, so the default/CI path exercises the encryption/egress/retention controls rather than first meeting them in production); a genuinely-synthetic dev/CI box must set `false` **explicitly** (a loud, audited opt-out), and a custom-named env must declare it | @@ -1534,7 +1542,7 @@ and a PHI weakening under **strict enforcement** (`enforcement = enforce`, the d > | Row | Reads as | Internal field with `[security]` absent | What an unconfigured PHI instance actually does | > |---|---|---|---| > | `block_unlisted_outbound` | `true` | `egress.deny_by_default = False` | the [`[egress]`](#egress) gate decides: with none of the six **counted** `allowed_*` lists `serve` **exits 2** (`allowed_smtp`/`allowed_direct` do not count); with ≥1 counted list it **flips deny-by-default on** for the transports you left empty | -> | `delete_message_bodies_after_days` | `30` | `retention.messages_days = 0` | under `enforcement = enforce` (the default) an unbounded window **refuses to start (exit 2)**; under `warn` each *unset* window is auto-bounded to 30 days; a synthetic instance keeps bodies forever | +> | `delete_message_bodies_after_days` | `30` | `retention.messages_days = 0` | the [`[retention]`](#retention) gate defaults each *unset* window to 30 days on a PHI instance under **both** enforcement dials; an **explicit** `0` refuses to start (exit 2) under `enforce` and warns under `warn`; a synthetic instance keeps bodies forever. This cell previously had the refuse / auto-bound split backwards | > > Neither is a silent fail-open — both paths end in a refusal or an audited flip, and `serve` > back-fills the `[security]` object from the resolved internal values before serving, so @@ -1604,13 +1612,19 @@ A **complete, startable** `messagefoundry.toml` for a loopback PHI instance on a run as `messagefoundry serve --config --env prod` (the active environment is required and has no default; `--env` is the CLI layer over `[ai].environment`, which is why it is not in the file). -**Three of these blocks exist only because a shipped serve gate refuses without them** — they are not -optional garnish. An earlier version of this example carried none of the three and would have hit -`exit 2` three times over. The four gates a stock PHI instance meets, and what satisfies each: +**Two of these blocks exist only because a shipped serve gate refuses without them** — they are not +optional garnish. An earlier version of this example carried neither and would have hit `exit 2` +twice over. `[retention]` is here for a different reason, given on its own line below. This paragraph +previously said three blocks and four refusing gates; the retention gate stopped refusing over an +unset window when the 30-day auto-bound moved to both enforcement dials. The gates a stock PHI +instance meets, and what satisfies each: **keyless PHI** → `MEFOR_STORE_ENCRYPTION_KEY` in the environment; **open egress** → at least one *counted* `[egress]` list (see the [`[egress]`](#egress) ⚠️ — `allowed_smtp` alone does not count); -**unbounded retention** → `[security].delete_message_bodies_after_days` **and** -`[retention].dead_letter_days`; **no security-notification channel** → the `[alerts]` SMTP transport. +**unbounded retention** → nothing you must configure to boot: `serve` defaults each *unset* PHI +window to 30 days rather than refusing, and only an **explicit** `0` is refused (see +[`[retention]`](#retention)) — set `[security].delete_message_bodies_after_days` and +`[retention].dead_letter_days` anyway, so the windows carry your site's numbers instead of the +engine's floor; **no security-notification channel** → the `[alerts]` SMTP transport. `[logging]` and `[api]` here are illustrative, not gate-required. `backend = "sqlserver"` also needs the `sqlserver` extra + ODBC Driver 18 installed (see the note under [`[store]`](#store--message-store--db)). @@ -1662,7 +1676,8 @@ forward_tls_ca_file = "C:/mefor/siem-ca.pem" # required for tls unless forward [retention] # The inbound-body window is [security].delete_message_bodies_after_days above — setting # messages_days here is REJECTED at load (ADR 0118). Only the plumbing keys stay in this section: -dead_letter_days = 90 # REQUIRED: an unbounded dead-letter window is its own exit 2 under enforce +dead_letter_days = 90 # RECOMMENDED: left unset this is auto-bounded to 30 days; an explicit 0 is + # an exit 2 under enforce. Set it so the window is your number, not the engine's # NOTE: vacuum_at / wal_checkpoint_seconds are SQLite-only and a documented NO-OP on this # backend = "sqlserver" store — space reclamation is a DBA operation there. Deliberately not set. ``` diff --git a/docs/PHI.md b/docs/PHI.md index a40b3c56..42166451 100644 --- a/docs/PHI.md +++ b/docs/PHI.md @@ -1052,12 +1052,31 @@ anywhere**. Config: [CONFIGURATION.md](CONFIGURATION.md#retention). **"Off by default" is no longer the whole truth on a PHI instance.** The raw `[retention]` fields do still default to `0`, but `serve` applies a posture gate on top of them: -- On a **non-enforcing PHI instance**, each **unset** PHI-body window is **auto-bounded to 30 days** - (secure-by-default) — an explicitly-set value, including an explicit `0`, is respected and only warns. -- On a **PHI instance under `[security].enforcement = enforce`** (the default) with either PHI-body - window unbounded, `serve` **refuses to start (exit code 2)**. +- On **any** PHI instance, under **both** `[security].enforcement` dials, each **unset** window that + carries an auto-bound is **defaulted to 30 days** at startup, and the defaulted settings are named + on stderr. The three are `[security].delete_message_bodies_after_days`, + `[retention].dead_letter_days` and `[retention].reference_snapshot_days`, generated from + [config/retention_classification.py](../messagefoundry/config/retention_classification.py) + (`auto_bounded_windows`) rather than listed at the gate. +- These bullets used to say the auto-bound applied only to a *non-enforcing* PHI instance, and that a + PHI instance under `enforcement = enforce` with a PHI-body window unbounded **refused to start (exit + code 2)**. Both halves were wrong about the shipped code: the auto-bound in + [`__main__.py`](../messagefoundry/__main__.py) is keyed on the retention opt-out alone + (`if not settings.retention.allow_unbounded_phi:`), not on the enforcement dial. A site that read + the old text and deliberately left a window unset — expecting the refusal to hold the boot until + someone chose a number — would instead get a started instance that begins purging PHI bodies at 30 + days. +- What survives is the fail-closed path for an **explicit** `0`. An explicitly-zeroed window is not + auto-bounded, and `serve` then **refuses to start (exit code 2)** under `enforcement = enforce`, or + warns and continues under `warn`. So "unbounded by accident" is still prevented; "unbounded by + inattention" becomes "30 days by inattention". +- The classified windows that carry **no** auto-bound are warned about at startup and left alone — + never silently defaulted and never refused over. `[retention].state_max_age_days` and + `[retention].search_preset_days` are the clearest case, because each keys on a timestamp that only + moves on a **write**, so a silent default would delete data a Handler is still reading; the others + are excluded for their own per-window reasons recorded alongside the classification. - The explicit, **audited** opt-out is `[security].allow_keeping_phi_indefinitely = true`, which - downgrades the refusal to a loud audited warning and suppresses the auto-bound. + suppresses the auto-bound **and** downgrades the refusal to a loud audited warning. - The canonical operator-facing home of the message-body window is now **`[security].delete_message_bodies_after_days`**; its *model* default is 30, but the desugar is **presence-gated** — only an EXPLICITLY-set switch is written through — so an **unset** @@ -1065,7 +1084,13 @@ still default to `0`, but `serve` applies a posture gate on top of them: refusal) is what actually bounds a PHI instance. An explicitly-set value writes through onto `[retention].messages_days`. `[retention].dead_letter_days` stays at its own home. -So a PHI instance cannot run with PHI-body retention "off" without a loud, audited opt-out. +So a PHI instance cannot run with PHI-body retention "off" **by accident** — an unset window is +bounded for you. It can still be run that way deliberately: an explicit `0` plus the audited opt-out +under `enforce`, or an explicit `0` alone under `warn`, which warns and starts. + +**Thirty days is the engine's floor against an accidentally unbounded window, not a retention +policy.** The engine picks that number; a deploying site with a retention obligation of its own must +set each window explicitly, because an auto-bounded window is a number nobody decided. **The pass itself.** It is **leader-gated twice** — at entry and again immediately before the purges, so a node demoted mid-pass never nulls PHI as a stale ex-leader. An optional between-phase wall-clock cap diff --git a/docs/SECURITY-LOOSENING.md b/docs/SECURITY-LOOSENING.md index 4515dc7f..03f0fcc7 100644 --- a/docs/SECURITY-LOOSENING.md +++ b/docs/SECURITY-LOOSENING.md @@ -216,10 +216,12 @@ trail. - **When acceptable:** a documented retention requirement that genuinely needs keep-forever, accepted in writing. - **Compensating controls:** a bounded window (e.g. 30 days); a startup **AUDIT** line records the override. -- **Still refused:** a **PHI** instance with an unbounded PHI-body window refuses under **strict enforcement** - (`enforcement = enforce`, the default) unless `allow_keeping_phi_indefinitely = true` (which downgrades the - refusal to a loud audited warning); at `enforcement = warn` a PHI instance auto-bounds each unset window to - 30 days. +- **Still refused:** a **PHI** instance whose PHI-body window is set **explicitly to `0`** refuses under + **strict enforcement** (`enforcement = enforce`, the default) unless `allow_keeping_phi_indefinitely = true` + (which downgrades the refusal to a loud audited warning). This bullet used to say the 30-day auto-bound of + an *unset* window happened only at `enforcement = warn`. It happens on **both** dials, so the refusal above + is reached only by an explicit `0` — or by the opt-out itself, which suppresses the auto-bound and therefore + leaves an unset window unbounded. ### `allow_single_factor_admin_when_exposed = true` — lift the strict-enforcement single-factor-admin refusal - **What you lose:** on a **PHI** instance under **strict enforcement** (`enforcement = enforce`, the default) @@ -295,7 +297,8 @@ trail. (`--allow-insecure-bind` / `MEFOR_ALLOW_INSECURE_TLS`) are **honoured** again. This reproduces the historical **non-production** PHI behaviour on a box that is otherwise strict-by-default: the cleartext off-box bind, open-egress, and single-factor-admin-at-exposure refusals downgrade to loud audited warnings, - and an unset PHI retention window auto-bounds to 30 days rather than refusing. Named once by + and an explicitly-zeroed PHI retention window warns rather than refusing. (The 30-day auto-bound of an + **unset** window is not part of this dial — it applies under `enforce` too.) Named once by `security_loosenings()` and in `GET /security/posture`. - **When acceptable:** a PHI **staging / pre-prod** box that must mirror production's *config* (so the encryption / egress / retention paths are exercised, not first met in production) but is deliberately run at