Skip to content
97 changes: 92 additions & 5 deletions docs/ASVS-L2-PHASE0-CHANGES.md

Large diffs are not rendered by default.

43 changes: 29 additions & 14 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -562,7 +562,7 @@ document ([SECURITY-DOCS-POLICY.md](SECURITY-DOCS-POLICY.md)).
| `password_min_length` | int | 15 | local-password policy — ASVS 5.0-aligned, length-first |
| `password_require_uppercase` / `password_require_lowercase` / `password_require_digit` / `password_require_symbol` | bool | `false` | character classes — **opt-in**, each independently (ASVS 5.0 forbids mandatory composition); turn one on only for a legacy standard that still mandates it |
| `password_check_breached` | bool | `true` | reject known common/breached passwords against a bundled offline top-10k list (no live HIBP call) |
| `password_check_context` | bool | `true` | reject passwords containing app/vendor/HL7 terms (e.g. `messagefoundry`, `mefor`, `hl7`, `corepoint`) |
| `password_check_context` | bool | `true` | reject a local password that **contains** any deny-list term — a case-insensitive substring test, anywhere in the value, not a whole-word or prefix match. The **twelve** terms are listed in full in [SECURITY.md](SECURITY.md) "Password policy"; an earlier revision of this row called them "app/vendor/HL7 terms" and gave four examples, which mis-stated the rule (five of the twelve are generic credential words unrelated to this application or to HL7). The list is fixed in code (`CONTEXT_WORDS` in [`auth/policy.py`](../messagefoundry/auth/policy.py)): **this flag turns the whole check on or off, and no setting adds or removes a term**, so a site needing its own vocabulary uses `password_breach_corpus_file` below — which matches the *whole* password, never a substring |
| `password_check_username` | bool | `true` | reject a password containing the user's **own username** (ASVS 6.2.11) |
| `password_breach_corpus_file` | path | — | optional path to a **larger offline breach corpus** that augments the bundled top-10k list (ASVS 6.2.12): a plaintext list **or** an HIBP-style SHA-1 hash export (`HASH[:count]` lines, auto-detected). Fully offline — still no live HIBP call. Use a curated subset, not the full ~40 GB HIBP set (it is loaded into memory). A path, not a secret |
| `lockout_threshold` | int | 5 | failed logins before lock (per account) |
Expand Down Expand Up @@ -716,11 +716,19 @@ disposition, and the audit trail stay intact — the Mirth Data-Pruner pattern);
`messages` row and never touches a body still in flight. The *row* survives; its PHI *columns* do not
— `messages.metadata` is nulled in the same statement as the body (ASVS 14.2.7). The raw `[retention]` fields still default to
`0`/`""` = keep/off, **but `serve` applies a posture gate on top of them, so retention is *not*
opt-in on a PHI instance**: under `[security].enforcement = enforce` (the default) an unbounded
`[security].delete_message_bodies_after_days` or `[retention].dead_letter_days` **refuses to start
(exit 2)**; on a non-enforcing PHI instance each *unset* window is auto-bounded to **30 days**. All
three built-in environment names (`dev`, `staging`, `prod`) derive PHI. The audited opt-out is
`[security].allow_keeping_phi_indefinitely = true`. See [PHI.md §8](PHI.md#8-retention--purge).
opt-in on a PHI instance**: each *unset* window that carries an auto-bound —
`[security].delete_message_bodies_after_days`, `[retention].dead_letter_days` and
`[retention].reference_snapshot_days` — is **defaulted to 30 days** at startup, under **both**
`[security].enforcement` dials, and the defaulted settings are named on stderr. A window set
**explicitly to `0`** is not defaulted: that **refuses to start (exit 2)** under `enforce`, and warns
under `warn`. This paragraph used to state the opposite split — refusal under `enforce`, auto-bound
only on a non-enforcing instance — which the shipped gate in
[`__main__.py`](../messagefoundry/__main__.py) refutes; an *unset* window has not refused since the
auto-bound moved to both dials. All three built-in environment names (`dev`, `staging`, `prod`)
derive PHI. The audited opt-out is `[security].allow_keeping_phi_indefinitely = true`, which
suppresses the auto-bound as well as the refusal. **Thirty days is the engine's floor against an
accidentally unbounded window, not your retention policy — set each window to the number your site
actually requires.** See [PHI.md §8](PHI.md#8-retention--purge).
| Key | Type | Default | Notes |
|---|---|---|---|
| `messages_days` | | | **→ moved to `[security].delete_message_bodies_after_days`** (ADR 0118) — set it there; no longer accepted in `[retention]`. |
Expand Down Expand Up @@ -1518,7 +1526,7 @@ and a PHI weakening under **strict enforcement** (`enforcement = enforce`, the d
| `sign_out_after_idle_minutes` | int | `30` | session idle timeout |
| `max_session_hours` | int | `12` | session absolute lifetime |
| `block_unlisted_outbound` | bool | `true` | deny-by-default egress — only allow-listed destinations send. **Leaving it unset does not apply `true`** — the internal flag stays `false` and the `[egress]` startup gate decides; see the note under this table |
| `delete_message_bodies_after_days` | int | `30` | bounded PHI-body retention; `0` = keep indefinitely (audited). **Leaving it unset does not apply 30** — the internal window stays `0` and the `[retention]` startup gate decides (refuse under `enforce`, auto-bound to 30 under `warn`); see the note under this table |
| `delete_message_bodies_after_days` | int | `30` | bounded PHI-body retention; `0` = keep indefinitely (audited). **Leaving it unset does not apply 30 through the desugar** — the internal window stays `0`, and the `[retention]` startup gate then defaults it to 30 days on a PHI instance under **either** enforcement dial. This row used to say the gate refuses under `enforce` and auto-bounds only under `warn`; it does not — only an **explicit** `0` reaches the refusal. See the note under this table |
| `allow_keeping_phi_indefinitely` | bool | `false` | audited escape: unbounded PHI retention |
| `audit_all_authorization_decisions` | bool | `false` | ePHI access is **always** audited regardless of this switch; this adds full *authorization-decision* tracing on top (off by default — forcing it on risks flooding the audit log). "Always audited" is about **coverage**, not about how hard those rows are to alter afterwards: the audit chain is only cryptographically tamper-*evident* on a **keyed** store, and its verify does not catch a truncated tail — see [`[integrity]`](#integrity) |
| `handles_real_patient_data` | bool | *derived* | the master data-class lever (was `[ai].data_class = "phi"`). Unset ⇒ derived from the environment name — **all three built-in names (`dev`/`staging`/`prod`) now derive PHI** ([ADR 0148](adr/0148-phi-default-posture-and-an-explicit-security-enforcement-level.md) GIVEN 1, so the default/CI path exercises the encryption/egress/retention controls rather than first meeting them in production); a genuinely-synthetic dev/CI box must set `false` **explicitly** (a loud, audited opt-out), and a custom-named env must declare it |
Expand All @@ -1534,7 +1542,7 @@ and a PHI weakening under **strict enforcement** (`enforcement = enforce`, the d
> | Row | Reads as | Internal field with `[security]` absent | What an unconfigured PHI instance actually does |
> |---|---|---|---|
> | `block_unlisted_outbound` | `true` | `egress.deny_by_default = False` | the [`[egress]`](#egress) gate decides: with none of the six **counted** `allowed_*` lists `serve` **exits 2** (`allowed_smtp`/`allowed_direct` do not count); with ≥1 counted list it **flips deny-by-default on** for the transports you left empty |
> | `delete_message_bodies_after_days` | `30` | `retention.messages_days = 0` | under `enforcement = enforce` (the default) an unbounded window **refuses to start (exit 2)**; under `warn` each *unset* window is auto-bounded to 30 days; a synthetic instance keeps bodies forever |
> | `delete_message_bodies_after_days` | `30` | `retention.messages_days = 0` | the [`[retention]`](#retention) gate defaults each *unset* window to 30 days on a PHI instance under **both** enforcement dials; an **explicit** `0` refuses to start (exit 2) under `enforce` and warns under `warn`; a synthetic instance keeps bodies forever. This cell previously had the refuse / auto-bound split backwards |
>
> Neither is a silent fail-open — both paths end in a refusal or an audited flip, and `serve`
> back-fills the `[security]` object from the resolved internal values before serving, so
Expand Down Expand Up @@ -1604,13 +1612,19 @@ A **complete, startable** `messagefoundry.toml` for a loopback PHI instance on a
run as `messagefoundry serve --config <dir> --env prod` (the active environment is required and has
no default; `--env` is the CLI layer over `[ai].environment`, which is why it is not in the file).

**Three of these blocks exist only because a shipped serve gate refuses without them** — they are not
optional garnish. An earlier version of this example carried none of the three and would have hit
`exit 2` three times over. The four gates a stock PHI instance meets, and what satisfies each:
**Two of these blocks exist only because a shipped serve gate refuses without them** — they are not
optional garnish. An earlier version of this example carried neither and would have hit `exit 2`
twice over. `[retention]` is here for a different reason, given on its own line below. This paragraph
previously said three blocks and four refusing gates; the retention gate stopped refusing over an
unset window when the 30-day auto-bound moved to both enforcement dials. The gates a stock PHI
instance meets, and what satisfies each:
**keyless PHI** → `MEFOR_STORE_ENCRYPTION_KEY` in the environment; **open egress** → at least one
*counted* `[egress]` list (see the [`[egress]`](#egress) ⚠️ — `allowed_smtp` alone does not count);
**unbounded retention** → `[security].delete_message_bodies_after_days` **and**
`[retention].dead_letter_days`; **no security-notification channel** → the `[alerts]` SMTP transport.
**unbounded retention** → nothing you must configure to boot: `serve` defaults each *unset* PHI
window to 30 days rather than refusing, and only an **explicit** `0` is refused (see
[`[retention]`](#retention)) — set `[security].delete_message_bodies_after_days` and
`[retention].dead_letter_days` anyway, so the windows carry your site's numbers instead of the
engine's floor; **no security-notification channel** → the `[alerts]` SMTP transport.
`[logging]` and `[api]` here are illustrative, not gate-required. `backend = "sqlserver"` also needs
the `sqlserver` extra + ODBC Driver 18 installed (see the note under [`[store]`](#store--message-store--db)).

Expand Down Expand Up @@ -1662,7 +1676,8 @@ forward_tls_ca_file = "C:/mefor/siem-ca.pem" # required for tls unless forward
[retention]
# The inbound-body window is [security].delete_message_bodies_after_days above — setting
# messages_days here is REJECTED at load (ADR 0118). Only the plumbing keys stay in this section:
dead_letter_days = 90 # REQUIRED: an unbounded dead-letter window is its own exit 2 under enforce
dead_letter_days = 90 # RECOMMENDED: left unset this is auto-bounded to 30 days; an explicit 0 is
# an exit 2 under enforce. Set it so the window is your number, not the engine's
# NOTE: vacuum_at / wal_checkpoint_seconds are SQLite-only and a documented NO-OP on this
# backend = "sqlserver" store — space reclamation is a DBA operation there. Deliberately not set.
```
Expand Down
33 changes: 26 additions & 7 deletions docs/CONNECTIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -715,14 +715,23 @@ at the source.

#### File handling & quarantine policy (ASVS 5.1.1)

MessageFoundry's file surface has three parts: the **directory sources** (the local `File(...)` and
remote `Sftp(...)`/`Ftp(...)` connectors) that ingest drop-directory files into the pipeline; the
MessageFoundry's file surface has **at least four** parts. This list is maintained by hand, so read it
as the current inventory and not as a closed set. The **directory sources** (the local `File(...)` and
remote `Sftp(...)`/`Ftp(...)` connectors) ingest drop-directory files into the pipeline; the
**opt-in HTTP uploaded-logs upload** (POST `/uploads` + the web-console delegate POST
`/ui/uploaded-logs/upload`, [ADR 0134](adr/0134-offline-uploaded-logs-viewer-connection-decoupled-upload-browse-resend-deletion-phi-at-rest-posture-stdlib-multipart.md))
for operator diagnostic logs; and the **attachment download** route (GET
carries operator diagnostic logs; the **attachment download** route (GET
`/messages/{message_id}/attachments/{attachment_id}`, [ADR 0105](adr/0105-streaming-very-large-hl7-attachments-detach-the-opaque-document-from-the-transformable-skeleton.md))
that serves a detached document back out. The **directory source's** handling of an untrusted drop
directory is fixed policy (the HTTP uploaded-logs surface has its own policy block below):
serves a detached document back out; and the **DICOM C-STORE SCP** (an inbound `DICOM(...)`,
[ADR 0025](adr/0025-dicom-codec-store-connectors.md)) receives whole objects **pushed by a remote
modality** over DIMSE. An earlier revision of this sentence said "three parts" and omitted the SCP;
that enumeration was wrong — the SCP is a receiver of remote-pushed content on the same footing as the
two HTTP routes. None of the drop-directory policy below applies to it: its size ceilings, peer
controls and transport security are connector settings documented under
[DICOM](#dicom--dicom-inbound-c-store-scp--outbound-c-store-scuc-echo-and-dicomweb-stow-rs-adr-0025),
and a deploying site must set them there rather than assume this block covers them. The **directory
source's** handling of an untrusted drop directory is fixed policy (the HTTP uploaded-logs surface has
its own policy block below):

- **Permitted type — the inbound's declared `content_type` (default `hl7v2`).** Files are selected by
the `pattern` glob (default `*.hl7`), and every candidate is **content-sniffed against that declared
Expand All @@ -739,7 +748,17 @@ directory is fixed policy (the HTTP uploaded-logs surface has its own policy blo
- **Maximum size.** `max_file_bytes` (default **16 MiB**, matching the MLLP frame cap). An oversize file
is rejected by a `stat()` **before** it is read into memory (OOM / DoS guard); `None`/`0` disables it.
- **Decompression is off by default; opt-in single-stream gzip is bomb-guarded** (ADR 0123). With no
`decompress=` set the connector reads raw bytes only and there is no unpacked-size surface. When
`decompress=` set the connector performs no decompression itself, so it materialises nothing beyond
`max_file_bytes` where that cap is set. An earlier revision went further and said there is "no
unpacked-size surface"; that was wrong — a file's *payload* can carry its own compressed stream. The
shipped case is a **Deflated Explicit VR LE** DICOM object, which the drop's content sniff accepts on
the `DICM` magic alone and which therefore reaches the pipeline with its inflated size unexamined.
That inflate is bounded where the object is unpacked rather than at ingest: at **16 MiB**, with no
per-connection knob, when a Router or Handler parses it (`guard_part10_deflate` in
`parsing/dicom/_inflate.py`, called from `DicomPeek.parse` and `DicomDataset.parse`), and at
`max_object_bytes` when an outbound C-STORE SCU forwards it. A site dropping DICOM into a watch
directory should size those two ceilings deliberately rather than read this bullet as saying no
unpacking happens. When
`decompress="gzip"` is enabled it gunzips each drop **before** the content sniff, the AV scan, and the
batch split (so all three see the real bytes), and `max_decompressed_bytes` (default 64 MiB) caps the
*decompressed* size — a decompression-bomb guard the compressed-only `max_file_bytes` cap cannot
Expand Down Expand Up @@ -1602,7 +1621,7 @@ MWL, Query/Retrieve (C-FIND/C-MOVE/C-GET), and pixel-data handling.
| `presentation_contexts` | `None` → SR + common image storage + Verification | the SOP classes the SCP negotiates (transfer syntaxes default to the standard set) |
| `calling_ae_allowlist` | `None` → any (subject to the IP gate) | only these calling AE titles may associate (fail-closed when set) |
| `require_called_ae_title` | `True` | a peer must address this engine's `ae_title` as the called AE |
| `max_object_bytes` | `134217728` (128 MiB) | reject a single C-STORE object larger than this **before** the durable commit (OOM/DoS guard) |
| `max_object_bytes` | `134217728` (128 MiB) | reject a single C-STORE object larger than this **before** the durable commit (OOM/DoS guard). It is **also** the ceiling for the pre-decode **inflate** of a *Deflated Explicit VR LE* object: the SCP bound-inflates the raw received Data Set before pydicom touches it, so an over-cap deflate bomb is a DIMSE failure and is never decoded or committed. Note that `0`/`None` does not simply widen this — it removes the object-size check entirely **and tightens** the inflate ceiling to the codec default of **16 MiB**, which is what the guard falls back to when no object cap is configured |
| `max_associations` | `10` | cap on concurrent inbound associations (connection-flood guard) |
| `max_pdu_size` | `16384` | cap one PDU's bytes (`0` = unbounded); DoS guard |
| `timeout_seconds` | `30.0` | ACSE/DIMSE/network timeout |
Expand Down
Loading
Loading