diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index e4c513d..ff0ff83 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -8,6 +8,16 @@ uv run pytest -q uv run ruff check . ``` +## Ownership boundaries + +Before adding general database or ORM infrastructure, check whether it belongs in a lower-level dependency: + +- [`orm-loader`](https://australiancancerdatanetwork.github.io/orm-loader/) owns domain-independent loading, serialization, and materialized-view lifecycle mechanics. +- `omop-alchemy` owns OMOP table models, clinical semantics, and OMOP-specific selectables and row grains. +- Consuming applications own view registries, dependency and rebuild policy, and deployment orchestration. + +Do not add materialized-view DDL, lifecycle helpers, or orchestration to `omop_alchemy`. + ## Opening a pull request 1. Apply **exactly one** label before merging: diff --git a/.gitignore b/.gitignore index 3e87184..e6640e9 100644 --- a/.gitignore +++ b/.gitignore @@ -74,4 +74,5 @@ notebooks/ .dockerignore docker/ tests/test_meds* -site/ \ No newline at end of file +site/ +_design/ \ No newline at end of file diff --git a/.importlinter b/.importlinter index 1078a5d..de9e387 100644 --- a/.importlinter +++ b/.importlinter @@ -9,4 +9,5 @@ layers = omop_alchemy.toolkit.analytics omop_alchemy.toolkit.episodes omop_alchemy.toolkit.core - omop_alchemy.cdm \ No newline at end of file + omop_alchemy.toolkit._utils + omop_alchemy.cdm diff --git a/README.md b/README.md index 7ddcb11..1f2ea72 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,8 @@ # OMOP Alchemy -**OMOP Alchemy** provides a canonical, typed, SQLAlchemy-first representation of the -[OHDSI OMOP Common Data Model (CDM)](https://ohdsi.github.io/CommonDataModel/). +**OMOP Alchemy** provides a canonical, typed, SQLAlchemy-first representation of the [OHDSI OMOP Common Data Model (CDM)](https://ohdsi.github.io/CommonDataModel/). -It is designed to support **research-ready analytics, validation, and exploration** -of OMOP data using modern Python tooling, without imposing ETL conventions or -execution-time side effects. +It is designed to support fluency for **research-ready analytics, validation, and exploration** of OMOP data using modern Python tooling, without imposing ETL conventions or execution-time side effects. --- @@ -16,30 +13,11 @@ OMOP Alchemy is intentionally: - **Declarative** Defines tables, columns, relationships, and constraints -- **SQLAlchemy-native** - Built for SQLAlchemy 2.x ORM usage - -- **Safe to import anywhere** - No implicit engine creation, no global state, no environment assumptions. - - **Typed and inspectable** Models are fully typed and introspectable for validation, tooling, and IDE support. - **Backend-agnostic** - Designed to work across PostgreSQL, SQLite, and other SQLAlchemy-supported databases. - ---- - -## What this package does *not* do - -OMOP Alchemy deliberately avoids: - -- Enforcing ETL conventions or data pipelines -- Auto-creating databases or loading vocabularies -- Imposing analytics frameworks or dashboards -- Making assumptions about deployment environments - -These concerns are intentionally left to downstream tooling. + Layered abstractions for adding in new supported backend behaviours. --- @@ -47,10 +25,9 @@ These concerns are intentionally left to downstream tooling. - SQLAlchemy ORM models for OMOP CDM tables - Explicit foreign key and relationship definitions -- Read-only *View* classes for safe navigation and analytics +- Lightweight mapper versions to provide simple model validation against CDM for use in ETL loops without side-effects or performance hit that can come from relationship instantiation within the runtime +- Read-only *View* classes for safe navigation and analytics that include complex multi-table objects such as conditions with their modifiers, episodes with their events - Domain validation helpers for OMOP concept integrity -- CSV loading utilities for controlled ingestion and testing -- Lightweight schema and model validation against CDM specs --- @@ -60,18 +37,18 @@ These concerns are intentionally left to downstream tooling. from omop_alchemy.cdm.model.vocabulary.concept import ConceptView concept = session.get(ConceptView, 320128) # Lung cancer -concept.domain.domain_id # "Condition" -concept.vocabulary.vocabulary_id # "SNOMED" -concept.is_standard # True +concept.domain.domain_id # "Condition" +concept.vocabulary.vocabulary_id # "SNOMED" +concept.is_standard # True ``` --- ## Status -This project is currently beta. +The core API under `cdm/` should be considered stable as of the 1.x release. -The API is stabilising, but some modules may change as real-world use cases expand. Feedback and issues are welcome. +The toolkit API is experimental and carries no compatibility guarantees. Feedback and issues are welcome. ### Some additional background @@ -82,9 +59,7 @@ This work builds on earlier research and tooling presented at the 2023 OHDSI APA ## Configuration -OMOP Alchemy reads all database connection and schema settings from -[oa-configurator](https://github.com/AustralianCancerDataNetwork/oa-configurator). -No `.env` files or `ENGINE` environment variables are needed. +OMOP Alchemy reads all database connection and schema settings from [oa-configurator](https://github.com/AustralianCancerDataNetwork/oa-configurator). No `.env` files or `ENGINE` environment variables are needed. Run once after installation: @@ -93,4 +68,4 @@ omop-config init omop-config configure omop_alchemy ``` -See [Configuration](docs/getting-started/configuration.md) for full details. \ No newline at end of file +See [Configuration](docs/getting-started/configuration.md) for full details. diff --git a/docs/advanced/fulltext.md b/docs/advanced/fulltext.md index 4a73a33..60876b5 100644 --- a/docs/advanced/fulltext.md +++ b/docs/advanced/fulltext.md @@ -1,7 +1,6 @@ # PostgreSQL Full-Text Search -OMOP Alchemy includes an **optional** PostgreSQL full-text search integration for -selected vocabulary text fields. +OMOP Alchemy includes an **optional** PostgreSQL full-text search integration for selected vocabulary text fields. This feature is deliberately bolt-on: @@ -32,19 +31,12 @@ backend.concept_synonym_name_tsvector_expression() These helpers return the best available expression for the configured environment: -- if the optional sidecar `tsvector` columns are registered in metadata, they return the - stored column -- otherwise they fall back to an inline computed PostgreSQL expression using - `to_tsvector(...)` +- if the optional sidecar `tsvector` columns are registered in metadata, they return the stored column +- otherwise they fall back to an inline computed PostgreSQL expression using `to_tsvector(...)` ### Example (PostgreSQL Documentation) -A tsvector value is a sorted list of distinct lexemes, which are words that have been normalized to merge different variants of the same word. -Sorting and duplicate-elimination are done automatically during input - - -A `tsvector` value is a sorted list of distinct lexemes (normalized word forms). -Sorting and duplicate elimination are applied automatically during input. +A `tsvector` value is a sorted list of distinct lexemes (normalised word forms). Sorting and duplicate elimination are applied automatically during input. ```sql SELECT 'a fat cat sat on a mat and ate a fat rat'::tsvector; @@ -63,8 +55,7 @@ omop-alchemy fulltext install omop-alchemy fulltext populate ``` -If your running Python process should use the stored sidecar columns through ORM -metadata, register them once at startup: +If your running Python process should use the stored sidecar columns through ORM metadata, register them once at startup: ```python from omop_alchemy.backends import resolve_backend @@ -73,8 +64,7 @@ backend = resolve_backend(engine) backend.register_fulltext_metadata() ``` -That is enough to activate the feature. The rest of this page explains when to use it -and how to operate it safely. +That is enough to activate the feature. The rest of this page explains when to use it and how to operate it safely. ## When To Use It @@ -101,12 +91,9 @@ Full-text search is useful, but it also introduces operational tradeoffs: - explicit backfill / refresh work - PostgreSQL-specific behavior -Many users only need occasional text matching and are perfectly fine with inline search -expressions. Others want fast repeated full-text lookups across large vocabularies and -are happy to manage the extra schema objects. +Many users only need occasional text matching and are perfectly fine with inline search expressions. Others want fast repeated full-text lookups across large vocabularies and are happy to manage the extra schema objects. -OMOP Alchemy therefore treats full-text sidecars as an **optional PostgreSQL -enhancement**, not as part of the core required OMOP schema. +OMOP Alchemy therefore treats full-text sidecars as an **optional PostgreSQL enhancement**, not as part of the core required OMOP schema. --- @@ -134,8 +121,7 @@ with Session(engine) as session: ) ``` -In practice you will often want the PostgreSQL full-text match operator rather than -equality: +In practice you will often want the PostgreSQL full-text match operator rather than equality: ```python vector = backend.concept_name_tsvector_expression() @@ -144,15 +130,13 @@ query = sa.func.plainto_tsquery("english", "edoxaban") stmt = sa.select(Concept).where(vector.op("@@")(query)) ``` -This mode is simple and portable at the library level, but PostgreSQL must compute the -vector expression at query time unless the planner can otherwise optimize it. +This mode is simple and portable at the library level, but PostgreSQL must compute the vector expression at query time unless the planner can otherwise optimize it. ### 2. Stored Sidecar Mode This mode adds real `tsvector` columns to the database and optionally GIN indexes. -Once installed and registered, the helper functions point at the stored columns instead -of recomputing vectors inline. +Once installed and registered, the helper functions point at the stored columns instead of recomputing vectors inline. This is the mode you want when: @@ -195,8 +179,7 @@ omop-alchemy fulltext drop ## Important Behavior -The current implementation uses **ordinary nullable sidecar `tsvector` columns**, not -generated columns and not trigger-managed columns. +The current implementation uses **ordinary nullable sidecar `tsvector` columns**, not generated columns and not trigger-managed columns. That means: @@ -204,8 +187,7 @@ That means: - `populate` backfills or refreshes the values - future data changes are **not** reflected automatically until you repopulate -This is a deliberate choice because it keeps the feature explicit and easier to manage -alongside bulk vocabulary loads. +This is a deliberate choice because it keeps the feature explicit and easier to manage alongside bulk vocabulary loads. --- @@ -245,8 +227,7 @@ The same idea applies to `backend.concept_synonym_name_tsvector_expression()`. ## Metadata Registration -If your process will use the stored sidecar columns directly, register them into the ORM -metadata: +If your process will use the stored sidecar columns directly, register them into the ORM metadata: ```python from omop_alchemy.backends import resolve_backend @@ -255,15 +236,13 @@ backend = resolve_backend(engine) backend.register_fulltext_metadata() ``` -If you later remove the columns from the database in the same process and want query -helpers to fall back cleanly again: +If you later remove the columns from the database in the same process and want query helpers to fall back cleanly again: ```python backend.unregister_fulltext_metadata() ``` -This only affects SQLAlchemy metadata in the current Python process. It does not alter -the database by itself. +This only affects SQLAlchemy metadata in the current Python process. It does not alter the database by itself. --- @@ -275,16 +254,13 @@ This feature is PostgreSQL-specific in its database form because it relies on: - PostgreSQL full-text query functions such as `to_tsvector` and `plainto_tsquery` - optional GIN indexes -The helper expressions can still be imported safely, but the sidecar install / populate / -drop lifecycle is only meaningful on PostgreSQL. +The helper expressions can still be imported safely, but the sidecar install / populate / drop lifecycle is only meaningful on PostgreSQL. --- -## Operational Gotchas +## Operational Notes: - treat the sidecar columns as **derived search state**, not source-of-truth data - if you bulk-load new vocabulary rows, rerun `omop-alchemy fulltext populate` -- if you use `reconcile-schema`, the sidecar columns and indexes are intentional - database additions outside the core OMOP schema -- GIN indexes can be expensive to build on large vocabularies, so plan that as a real - maintenance operation rather than a trivial toggle \ No newline at end of file +- if you use `reconcile-schema`, the sidecar columns and indexes are intentional database additions outside the core OMOP schema +- GIN indexes can be expensive to build on large vocabularies, so plan that as a real maintenance operation rather than a trivial toggle diff --git a/docs/advanced/timelines.md b/docs/advanced/timelines.md index c18a16d..482cba0 100644 --- a/docs/advanced/timelines.md +++ b/docs/advanced/timelines.md @@ -1,10 +1,8 @@ # Patient Timelines -OMOP Alchemy includes a lightweight timeline layer that projects OMOP CDM ORM objects -into a **unified, time-ordered event stream** per patient. +OMOP Alchemy includes a lightweight timeline layer that projects OMOP CDM ORM objects into a **unified, time-ordered event stream** per patient. -It is primarily intended for feature construction and exploratory analysis — not for -production query pipelines where raw SQLAlchemy queries are more appropriate. +It is primarily intended for feature construction and exploratory analysis — not for production query pipelines where raw SQLAlchemy queries are more appropriate. --- @@ -12,8 +10,7 @@ production query pipelines where raw SQLAlchemy queries are more appropriate. ### `EventTime` -A canonical temporal representation. Every clinical event has a start datetime; an end -datetime is optional. The `kind` property returns `"point"` or `"interval"`. +A canonical temporal representation. Every clinical event has a start datetime; an end datetime is optional. The `kind` property returns `"point"` or `"interval"`. ::: omop_alchemy.toolkit.core.timeline.event_timeline.EventTime @@ -29,9 +26,7 @@ The value associated with a clinical event — numeric, concept, string, or none ### `EventMapping` -Declares which ORM fields supply the concept, start/end datetimes, and value for a -particular CDM table. Subclasses of `ClinicalEvent` set `_mapping` to an `EventMapping` -instance at class level. +Declares which ORM fields supply the concept, start/end datetimes, and value for a particular CDM table. `EventMapping.from_model()` derives identity, source, concept and start fields from the shared CDM metadata used by canonical SQL projections. It also infers independent interval endpoints from that metadata: Measurement and Observation alias their one date column in the target API and remain point events in the timeline. Timeline classes add value and display-specific fields. Explicit endpoint strings override inference; explicit `None` disables the corresponding inferred endpoint. ::: omop_alchemy.toolkit.core.timeline.event_timeline.EventMapping @@ -39,9 +34,7 @@ instance at class level. ## The `ClinicalEvent` mixin -`ClinicalEvent` is a mixin that adds timeline behaviour to any CDM ORM class. It reads -`_mapping` to implement `event_time`, `event_value`, `event_metadata`, `to_dict`, and -`to_json`. +`ClinicalEvent` is a mixin that adds timeline behaviour to any CDM ORM class. It implements the shared `toolkit.core.events.ClinicalEventRow` identity and projection fields, then reads `_mapping` to add `event_time`, `event_value`, `event_metadata`, `to_dict`, and `to_json`. The shared core contract keeps timeline events and SQL event projections aligned without making `core.timeline` import the higher-level episode package. ::: omop_alchemy.toolkit.core.timeline.event_timeline.ClinicalEvent @@ -49,13 +42,14 @@ instance at class level. ## Concrete event classes -Three CDM tables are pre-wired with `EventMapping`s: +Four CDM tables are pre-wired with `EventMapping`s: | Class | CDM table | Concept field | Value fields | |-------|-----------|---------------|--------------| | `Condition_Event` | `condition_occurrence` | `condition_concept_id` | — | -| `Measurement_Event` | `measurement` | `measurement_concept_id` | `value_as_number`, `value_as_concept_id`, `value_as_string` | +| `Measurement_Event` | `measurement` | `measurement_concept_id` | `value_as_concept_id`, `value_as_number` | | `Drug_Exposure_Event` | `drug_exposure` | `drug_concept_id` | `quantity` | +| `Observation_Event` | `observation` | `observation_concept_id` | `value_as_concept_id`, `value_as_number`, `value_as_string` | ::: omop_alchemy.toolkit.core.timeline.event_timeline.Condition_Event @@ -63,13 +57,13 @@ Three CDM tables are pre-wired with `EventMapping`s: ::: omop_alchemy.toolkit.core.timeline.event_timeline.Drug_Exposure_Event +::: omop_alchemy.toolkit.core.timeline.event_timeline.Observation_Event + --- ## `Person_Timeline` -Extends the `Person` ORM class with `.events` and `.timeline` properties. Requires an -active SQLAlchemy session (i.e. the object must have been loaded from a session, not -constructed in memory). +Extends the `Person` ORM class with `.events` and `.timeline` properties. Requires an active SQLAlchemy session (i.e. the object must have been loaded from a session, not constructed in memory). ::: omop_alchemy.toolkit.core.timeline.event_timeline.Person_Timeline @@ -88,21 +82,18 @@ with Session(engine) as session: print(event.to_dict()) ``` +Serialized timeline events include `event_id`, `event_source_table`, `event_field_concept_id` and `event_concept_id`. Direct `EventMapping` construction requires `event_id_field`, `event_source_table` and `event_field_concept_id`; `EventMapping.from_model()` derives these fields from model metadata. + --- ## Extending to new tables -To add a new CDM table to the timeline, subclass both `ClinicalEvent` and the target ORM -class and set `_mapping`: +To add a supported CDM table to the timeline, subclass both `ClinicalEvent` and the target ORM class and build `_mapping` from its Core metadata: ```python from omop_alchemy.toolkit.core.timeline.event_timeline import ClinicalEvent, EventMapping from omop_alchemy.cdm.model.clinical import Procedure_Occurrence -class Procedure_Event(Procedure_Occurrence, ClinicalEvent): - _mapping = EventMapping( - concept_field="procedure_concept_id", - start_date_field="procedure_date", - start_datetime_field="procedure_datetime", - ) +class Procedure_Event(ClinicalEvent, Procedure_Occurrence): + _mapping = EventMapping.from_model(Procedure_Occurrence) ``` diff --git a/docs/advanced/vocabulary_load_performance.md b/docs/advanced/vocabulary_load_performance.md index 9ec4703..733efcc 100644 --- a/docs/advanced/vocabulary_load_performance.md +++ b/docs/advanced/vocabulary_load_performance.md @@ -23,11 +23,11 @@ Empirical timing on an 8 GB RAM dev machine with `concept_relationship` (~56 M r | Configuration | Total load time | |---------------|----------------| -| `--merge-batch-size 1_000_000` (old default) | ~120 min | +| `--merge-batch-size 1_000_000` | ~120 min | | `--merge-batch-size 20_000_000` | ~80 min | | No pagination (default) | **~40 min** | -The staging index build alone on a 56 M-row table adds 10–15 minutes regardless of batch size. **The default is now `None` (no pagination).** Only set `--merge-batch-size` if your system cannot hold the full merge in a single transaction. +The staging index build alone on a 56 M-row table adds 10–15 minutes regardless of batch size. **The default is `None` (no pagination).** Only set `--merge-batch-size` if your system cannot hold the full merge in a single transaction. > **Warning:** Setting `--merge-batch-size` to a large number to "avoid" pagination does not help if that number is still smaller than the largest table. For `concept_relationship` (~56 M rows), any value below 56 M will trigger the index build. If you need pagination, set it to your actual memory limit; if you don't, leave it unset. @@ -37,7 +37,7 @@ The next biggest bottleneck after pagination is `synchronous_commit=on` (the Pos ### Recommended settings -Apply these via `postgresql.conf` or `-c` flags on whatever PostgreSQL instance you're loading into (per-package `docker-compose.yaml` files no longer exist; Docker orchestration for the OMOP stack now happens at the workspace root). +Apply these via `postgresql.conf` or `-c` flags on the PostgreSQL instance you're loading into. **8 GB host:** ``` diff --git a/docs/api/architecture.md b/docs/api/architecture.md index aee5eb4..54a8125 100644 --- a/docs/api/architecture.md +++ b/docs/api/architecture.md @@ -2,8 +2,7 @@ OMOP Alchemy is built as a **deliberately layered system**. -Each layer adds capability while preserving the guarantees of the layer below it. -Responsibilities flow *downward*; semantic intent flows *upward*. +Each layer adds capability while preserving the guarantees of the layers beneath it. The core layers build upward from generic infrastructure to OMOP-aware views; Toolkit and Semantic Validation build on those views independently. The result is a system that is: @@ -17,161 +16,86 @@ The result is a system that is: ```mermaid -flowchart TD +flowchart BT subgraph L0["orm-loader"] L0a["CSVLoadableTableInterface"] L0b["SerialisableTableInterface"] L0c["Bulk load & casting helpers"] + L0d["Materialized-view lifecycle"] end - subgraph L1["cdm.base"] - L1a["CDMTableBase"] - L1b["Column helpers"] - L1c["Structural mixins"] - L1d["ReferenceContext"] - L1e["DomainValidation primitives"] + subgraph CDM["omop_alchemy.cdm"] + direction BT + subgraph L1["cdm.base"] + L1a["CDMTableBase"] + L1b["Column helpers"] + L1c["Structural mixins"] + L1d["ReferenceContext"] + L1e["DomainValidation primitives"] + end + + subgraph L2["cdm.models"] + L2a["Concrete CDM tables"] + L2b["@cdm_table"] + L2c["OMOP-compliant schemas"] + end + + subgraph L3["Views & Contexts"] + L3a["Reference contexts"] + L3b["Derived properties"] + L3c["Hybrid expressions"] + end end - subgraph L2["cdm.models"] - L2a["Concrete CDM tables"] - L2b["@cdm_table"] - L2c["OMOP-compliant schemas"] + subgraph L4["Toolkit"] + L4a["Standard queries, composition & exploration"] end - subgraph L3["Views & Contexts"] - L3a["Reference contexts"] - L3b["Derived properties"] - L3c["Hybrid expressions"] + subgraph L5["Semantic Validation"] + L5a["ExpectedDomain"] + L5b["DomainRule"] + L5c["Runtime domain checks"] end - subgraph L4["Semantic Validation"] - L4a["ExpectedDomain"] - L4b["DomainRule"] - L4c["Runtime domain checks"] - end - - L0 --> L1 - L1 --> L2 - L2 --> L3 - L3 --> L4 + L0 -->|infrastructure| L1 + L1 -->|OMOP structure| L2 + L2 -->|views & navigation| L3 + L3 -->|reusable queries| L4 + L3 -->|semantic checks| L5 + + classDef infrastructure fill:#e8f1fb,stroke:#2b6cb0,stroke-width:1.5px,color:#17324d + classDef cdmLayer fill:#eaf6ef,stroke:#2f855a,stroke-width:1.5px,color:#193b29 + classDef toolkitLayer fill:#fff6df,stroke:#b7791f,stroke-width:1.5px,color:#4a3210 + classDef validationLayer fill:#f3eaff,stroke:#805ad5,stroke-width:1.5px,color:#33224d + + class L0a,L0b,L0c,L0d infrastructure + class L1a,L1b,L1c,L1d,L1e,L2a,L2b,L2c,L3a,L3b,L3c cdmLayer + class L4a toolkitLayer + class L5a,L5b,L5c validationLayer + + style CDM fill:#f8fafc,stroke:#64748b,stroke-width:2px,stroke-dasharray:5 5 + style L0 fill:#f5faff,stroke:#2b6cb0,stroke-width:1.5px + style L1 fill:#f3fbf5,stroke:#2f855a,stroke-width:1.5px + style L2 fill:#f3fbf5,stroke:#2f855a,stroke-width:1.5px + style L3 fill:#f3fbf5,stroke:#2f855a,stroke-width:1.5px + style L4 fill:#fffaf0,stroke:#b7791f,stroke-width:1.5px + style L5 fill:#faf7ff,stroke:#805ad5,stroke-width:1.5px ``` ### Layer responsibilities -#### orm-loader (L0) - -Purpose: ingestion and infrastructure - -This layer provides: - -* CSV loading -* bulk inserts -* type casting -* serialization helpers - -It is deliberately domain-agnostic. - -If something understands OMOP concepts, vocabularies, or clinical meaning, it does not belong here. - -Examples: - -* [CSVLoadableTableInterface](https://australiancancerdatanetwork.github.io/orm-loader/loaders/) -* [SerialisableTableInterface](https://australiancancerdatanetwork.github.io/orm-loader/tables/serialisable_table/) - -#### cdm.base (L1) - -Purpose: structural OMOP semantics - -This layer encodes: - -* common OMOP table structure -* column patterns (required vs optional) -* reusable mixins -* reference relationship mechanics -* domain validation primitives - -This is where OMOP’s shape lives, but not its analytical meaning. - -Examples: - -* [CDMTableBase](./base.md) -* [PersonScoped](./columns.md) -* [ReferenceContext](./relationships.md) - -This layer answers: - -“What does a valid OMOP table look like?” - -#### cdm.models (L2) - -Purpose: concrete OMOP tables - -This layer defines: - -* actual CDM tables -* exact column layouts -* primary keys and foreign keys -* official OMOP schemas - -Classes in this layer: - -* are safe for ETL -* are safe for bulk loading -* avoid eager relationships -* avoid analytical helpers - -Examples: - -[Person](../models/clinical/person.md) - -#### Views & Contexts (L3) - -Purpose: navigation and analysis - -This layer adds: - -* reference relationships -* derived properties -* hybrid expressions -* query-friendly helpers - -Examples: - -[PersonContext](../models/clinical/person.md) -[PersonView](../models/clinical/person.md) - -Views are designed for interactive use, not ingestion. - -> *Tables are for pipelines. Views are for people.* - -#### Semantic Validation (L4) - -Purpose: make semantic expectations explicit - -This layer introduces: - -* declared domain expectations -* inspectable rules -* runtime, advisory checks - -Examples: - -[ExpectedDomain](../validation/index.md) - -This layer answers: - -> “Does this object reference the kinds of concepts I think it does?” - -Validation here is: - -* non-blocking -* non-mutating -* safe to skip -* safe to run interactively +| Layer | Purpose | Responsibilities | Boundary and examples | +| --- | --- | --- | --- | +| **orm-loader (L0)** | Ingestion and infrastructure | CSV loading; bulk inserts; type casting; serialization; materialized-view definition and lifecycle operations | Domain-agnostic: it does not understand OMOP concepts, vocabularies, or clinical meaning. OMOP Alchemy may provide a selectable and logical row identity, while applications own view collections, dependency policy, and deployment. [CSVLoadableTableInterface](https://australiancancerdatanetwork.github.io/orm-loader/loaders/)
[SerialisableTableInterface](https://australiancancerdatanetwork.github.io/orm-loader/tables/serialisable_table/)
[Materialized views](https://australiancancerdatanetwork.github.io/orm-loader/tables/mat_view/) | +| **cdm.base (L1)** | Structural OMOP semantics | Common table structure; required and optional column patterns; reusable mixins; reference relationship mechanics; domain validation primitives | Defines OMOP’s shape, not its analytical meaning. Answers: “What does a valid OMOP table look like?” [CDMTableBase](./base.md); [PersonScoped](./columns.md); [ReferenceContext](./relationships.md) | +| **cdm.models (L2)** | Concrete OMOP tables | Actual CDM tables; exact column layouts; primary and foreign keys; official OMOP schemas | Safe for ETL and bulk loading; avoids eager relationships and analytical helpers. Example: [Person](../models/clinical/person.md) | +| **Views & Contexts (L3)** | Navigation and analysis | Reference relationships; derived properties; hybrid expressions; query-friendly helpers | Designed for interactive use, not ingestion: *tables are for pipelines; views are for people.* [PersonContext](../models/clinical/person.md); [PersonView](../models/clinical/person.md) | +| **Toolkit (L4)** | Reusable clinical queries and composition | Standard query contracts; vocabulary and concept resolution; event and episode composition; reusable analytical projections | Consumes CDM models and views, adding interpretation and retrieval rules without replacing models or hiding database access. [Toolkit overview](../toolkit/index.md); [Query contracts](../toolkit/query-contracts.md) | +| **Semantic Validation (L5)** | Explicit semantic expectations | Declared domain expectations; inspectable rules; runtime advisory checks | Non-blocking and non-mutating; safe to skip or run interactively. Answers: “Does this object reference the kinds of concepts I think it does?” [ExpectedDomain](../validation/index.md) | ### Directional guarantees -The stack is intentionally one-directional: +The dependency direction is intentionally one-way: higher layers may use lower layers, but lower layers never import higher layers. In the diagram, arrows show capability building upward from the foundation; Toolkit and Semantic Validation are peers rather than dependencies of one another. * lower layers never import higher layers * ETL code never depends on analytical helpers diff --git a/docs/api/base.md b/docs/api/base.md index b11d9ef..9888871 100644 --- a/docs/api/base.md +++ b/docs/api/base.md @@ -1,10 +1,8 @@ # Base Tables -Base tables define the **foundation of all concrete OMOP CDM models** -in OMOP Alchemy. +Base tables define the **foundation of all concrete OMOP CDM models** in OMOP Alchemy. -They establish the minimum structural and behavioral contract that -distinguishes a real CDM table from: +They establish the minimum structural and behavioral contract that distinguishes a real CDM table from: - mixins - views @@ -37,8 +35,7 @@ Those concerns live elsewhere. ## Relationship to `orm-loader` -`CDMTableBase` builds directly on infrastructure provided by -`orm-loader`. +`CDMTableBase` builds directly on infrastructure provided by `orm-loader`. Specifically, it inherits: @@ -58,8 +55,7 @@ This separation is intentional: ## `CDMTableBase` -The `CDMTableBase` class is the common ancestor for all concrete -OMOP CDM tables. +The `CDMTableBase` class is the common ancestor for all concrete OMOP CDM tables. ::: omop_alchemy.cdm.base.cdm_table_base.CDMTableBase options: diff --git a/docs/api/columns.md b/docs/api/columns.md index 85bb52d..5d288c4 100644 --- a/docs/api/columns.md +++ b/docs/api/columns.md @@ -1,7 +1,6 @@ # Columns & Structural Mixins -OMOP Alchemy provides a small set of **column helpers and mixins** -that encode recurring OMOP CDM patterns directly into ORM structure. +OMOP Alchemy provides a small set of **column helpers and mixins** that encode recurring OMOP CDM patterns directly into ORM structure. These utilities exist to: @@ -10,23 +9,19 @@ These utilities exist to: - keep table definitions readable - align ORM structure with CDM specifications -They are **structural**, not analytical: -they describe *how data is shaped*, not *what it means*. +They are **structural**, not analytical: they describe *how data is shaped*, not *what it means*. --- ## Column helper functions -Column helpers wrap common OMOP column patterns into -small, intention-revealing factory functions. +Column helpers wrap common OMOP column patterns into small, intention-revealing factory functions. -They are thin wrappers around `sqlalchemy.orm.mapped_column` -with defaults chosen to match the CDM Field-Level specifications. +They are thin wrappers around `sqlalchemy.orm.mapped_column` with defaults chosen to match the CDM Field-Level specifications. ### Concept foreign keys -OMOP relies heavily on concept identifiers, with specific semantics -around nullability and unknown values. +OMOP relies heavily on concept identifiers, with specific semantics around nullability and unknown values. ::: omop_alchemy.cdm.base.column_helpers.required_concept_fk options: @@ -40,8 +35,7 @@ around nullability and unknown values. ### Convenience wrappers -These helpers exist primarily for consistency and readability -when defining large tables with many fields. +These helpers exist primarily for consistency and readability when defining large tables with many fields. ::: omop_alchemy.cdm.base.column_helpers.optional_fk options: @@ -59,8 +53,7 @@ when defining large tables with many fields. ## Structural mixins -Structural mixins encode **table-level OMOP patterns** that recur -across multiple CDM tables. +Structural mixins encode **table-level OMOP patterns** that recur across multiple CDM tables. ::: omop_alchemy.cdm.base.column_mixins.PersonScoped options: diff --git a/docs/api/index.md b/docs/api/index.md index 7321e6d..244096c 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -1,11 +1,8 @@ # API Reference +This section documents the **core authoring primitives** used to define OMOP CDM models in OMOP Alchemy. -This section documents the **core authoring primitives** used to define -OMOP CDM models in OMOP Alchemy. - -These APIs are intentionally **low-level, explicit, and composable**. -They are designed for *model authors*, not end-users or analysts. +These APIs are intentionally **low-level, explicit, and composable**. They are designed for *model authors*, not end-users or analysts. If you are: @@ -40,6 +37,7 @@ The APIs documented here follow a few consistent principles: All modules are import-safe and side-effect free. This makes the API suitable for: + - schema inspection - validation tooling - static analysis @@ -54,11 +52,9 @@ Layered architecture specification is described in [Architecture](./architecture ## Base table infrastructure -At the core of OMOP Alchemy is a small number of base classes that define -what it means to be a CDM table. +At the core of OMOP Alchemy is a small number of base classes that define what it means to be a CDM table. -These classes integrate with lower-level infrastructure (i.e. -`orm-loader`) but remain OMOP-specific. +These classes integrate with lower-level infrastructure (i.e. `orm-loader`) but remain OMOP-specific. **[Base tables](base.md)** @@ -77,15 +73,16 @@ Column helpers provide **named, intention-revealing shortcuts** for these patter ## Structural mixins -Many OMOP tables share structural semantics: +Many OMOP tables share structural semantics for example: - person-scoped records - dated events - value-typed observations - unit concepts - health system attribution +- accepted modifiable targets / sources -Mixins encode these patterns once, and make them reusable and inspectable. +Don't define this behaviour directly unless it is truly a new pattern - instead you should use the available mixins, which encode these patterns once, and make them reusable and inspectable. **[Column mixins](columns.md)** @@ -93,14 +90,14 @@ Mixins encode these patterns once, and make them reusable and inspectable. ## Typing and semantic contracts -OMOP Alchemy makes heavy use of Python typing to express -*semantic expectations*: +OMOP Alchemy makes heavy use of Python typing to express *semantic expectations*: - “this object has a concept_id” - “this table participates in domain validation” - “this is a clinical event” These protocols support: + - static type checking - IDE assistance - tooling and validation layers @@ -111,13 +108,12 @@ These protocols support: ## Relationship to other layers -This API layer sits *above* generic ORM infrastructure -and *below* analytical or validation tooling. +This API layer sits *above* generic ORM infrastructure and *below* analytical or validation tooling. | Layer | Responsibility | |------|---------------| | Database | Physical storage, constraints | | orm-loader | Generic table loading, serialization | -| OMOP Alchemy API | OMOP-specific structure & semantics | -| Validation | Domain and semantic checks | -| Analytics | Queries, cohorts, exploration | \ No newline at end of file +| **OMOP Alchemy API** | **OMOP-specific structure & semantics** | +| **OMOP Alchemy Toolkit** | **Standard, generalisable queries, composition, exploration** | +| Validation | Domain and semantic checks | \ No newline at end of file diff --git a/docs/api/query.md b/docs/api/query.md index e85bb48..5d35fc9 100644 --- a/docs/api/query.md +++ b/docs/api/query.md @@ -1,14 +1,8 @@ # Query Filtering -`omop_alchemy.cdm.query` provides `ConceptFilter`, a shared, reusable way to -filter CDM `concept`-table queries by domain, vocabulary, concept ID, and -standard/active status, with an optional row-count limit. +`omop_alchemy.cdm.query` provides `ConceptFilter`, a shared, reusable way to filter CDM `concept`-table queries by domain, vocabulary, concept ID, and standard/active status, with an optional row-count limit. -It exists so that packages consuming OMOP Alchemy (e.g. `omop-emb`, `omop-graph`) -don't each need to reimplement the same filtering logic against their own copy -of `Concept`'s column names — since this package owns the `Concept` model -directly, the filter can reference real columns rather than duck-typing against -an opaquely-imported table. +It exists so that downstream packages do not each need to reimplement the same filtering logic against their own copy of `Concept`'s column names. Since this package owns the `Concept` model directly, the filter can reference real columns rather than duck-typing against an opaquely imported table. ```python from sqlalchemy import select diff --git a/docs/api/relationships.md b/docs/api/relationships.md index e3ac2cc..6c9aada 100644 --- a/docs/api/relationships.md +++ b/docs/api/relationships.md @@ -2,15 +2,13 @@ OMOP Alchemy takes a **deliberately conservative approach to ORM relationships**. -Rather than eagerly wiring every foreign key into a bidirectional relationship, -it distinguishes between: +Rather than eagerly wiring every foreign key into a bidirectional relationship, it distinguishes between: - **structural foreign keys** (always present in tables) - **reference lookups** (read-only joins for navigation) - **analytical relationships** (used in views, not ETL) -This separation keeps core tables simple, predictable, and fast to load, -while still enabling rich, expressive navigation when you need it. +This separation keeps core tables simple, predictable, and fast to load, while still enabling rich, expressive navigation when you need it. --- @@ -40,8 +38,7 @@ OMOP Alchemy addresses this by introducing **Reference Contexts**. ## The core idea -Instead of defining relationships directly on a table class, -OMOP Alchemy encourages a **three-layer pattern**: +Instead of defining relationships directly on a table class, OMOP Alchemy encourages a **three-layer pattern**: 1. **Table** – structural definition only 2. **Context** – reference relationships @@ -160,8 +157,7 @@ Vocabulary tables (Concept, Domain, Vocabulary, etc.) are: * stable * not owned by fact tables -Allowing mutation through ORM relationships would blur those boundaries -and make ETL behavior harder to reason about. +Allowing mutation through ORM relationships would blur those boundaries and make ETL behavior harder to reason about. ### Performance considerations diff --git a/docs/api/typing.md b/docs/api/typing.md index 4216c40..ad7cabc 100644 --- a/docs/api/typing.md +++ b/docs/api/typing.md @@ -1,7 +1,6 @@ # Typing -OMOP Alchemy exposes a set of **Protocols and typed containers** for code that needs to -interact with CDM classes without coupling to specific ORM implementations. +OMOP Alchemy exposes a set of **Protocols and typed containers** for code that needs to interact with CDM classes without coupling to specific ORM implementations. These live in two modules: @@ -52,9 +51,7 @@ Satisfied by any object with an integer `episode_id` attribute. ### `DomainSemanticTable` -Structural protocol for CDM ORM classes that participate in domain validation. A class -satisfies this protocol if it has `__tablename__`, `__mapper__`, `__expected_domains__`, -and a `collect_domain_rules()` classmethod. +Structural protocol for CDM ORM classes that participate in domain validation. A class satisfies this protocol if it has `__tablename__`, `__mapper__`, `__expected_domains__`, and a `collect_domain_rules()` classmethod. ::: omop_alchemy.cdm.base.typing.DomainSemanticTable options: @@ -68,8 +65,7 @@ and a `collect_domain_rules()` classmethod. ### `ConceptRow` -A frozen dataclass representing the core fields of a concept lookup row. Used where a -lightweight, hashable concept record is preferable to a full ORM object. +A frozen dataclass representing the core fields of a concept lookup row. Used where a lightweight, hashable concept record is preferable to a full ORM object. ::: omop_alchemy.cdm.model.typing.ConceptRow options: diff --git a/docs/assets/images/oa-configure.png b/docs/assets/images/oa-configure.png new file mode 100644 index 0000000..ee23348 Binary files /dev/null and b/docs/assets/images/oa-configure.png differ diff --git a/docs/assets/images/oa-fulltext.png b/docs/assets/images/oa-fulltext.png new file mode 100644 index 0000000..6c0e7be Binary files /dev/null and b/docs/assets/images/oa-fulltext.png differ diff --git a/docs/assets/images/oa-info.png b/docs/assets/images/oa-info.png new file mode 100644 index 0000000..c55d233 Binary files /dev/null and b/docs/assets/images/oa-info.png differ diff --git a/docs/getting-started/configuration.md b/docs/getting-started/configuration.md index 8483ced..20241ca 100644 --- a/docs/getting-started/configuration.md +++ b/docs/getting-started/configuration.md @@ -1,23 +1,22 @@ # Configuration -OMOP_Alchemy reads all database connection and schema settings from -[oa_configurator](https://github.com/AustralianCancerDataNetwork/oa-configurator) — no -`.env` files or `ENGINE` environment variables needed. +OMOP_Alchemy reads all database connection and schema settings from [oa_configurator](https://github.com/AustralianCancerDataNetwork/oa-configurator) ## Minimal config -Run the interactive configure command to set up the CDM database connection and write -`~/.config/omop/config.toml`: +Run the interactive configure command to set up the CDM database connection and write `~/.config/omop/config.toml`: ```bash omop-config configure omop_alchemy ``` -This prompts for connection details (host, dialect, credentials) and schema name, then -saves them under the canonical database name `cdm_db` that all OMOP stack packages -recognise. +This prompts for connection details (host, dialect, credentials) and schema name, then saves them under the canonical database name `cdm_db` that all OMOP stack packages recognise. -The resulting TOML looks like: +The default location for this file is `~/.config/omop/config.toml` + +![configure](../assets/images/oa-configure.png) + +The resulting TOML will look like: ```toml [connections.cdm] @@ -27,6 +26,7 @@ port = 5432 user = "omop" password = "changeme" database_name = "omop_cdm" +test_only = false [databases.cdm_db] kind = "cdm" @@ -37,7 +37,7 @@ schema_name = "omop" cdm_db = "cdm_db" ``` -You can also write or edit this file manually. +You can also write or edit this file manually. It follows the `oa-configurator` pattern of [physical]->[logical] resource definition, where one connection may serve multiple databases, and each application may define its own database resource, or choose to cross reference an existing one that will be resolved upon connection in the consuming application. ## Vocabulary loading @@ -49,11 +49,7 @@ cdm_db = "cdm_db" athena_source_path = "/path/to/athena/csvs" ``` -Or set it interactively: - -```bash -omop-config configure omop_alchemy -``` +This may be edited directly, set interactively in the `omop-config configure omop_alchemy` process, or set directly using the CLI. ## Verify @@ -61,22 +57,20 @@ omop-config configure omop_alchemy omop-alchemy info ``` -This prints the resolved config file path, connection details, and schema. A successful -run confirms that OMOP_Alchemy can reach your database. +This prints the resolved config file path, connection details, and schema. A successful run confirms that OMOP_Alchemy can reach your database. + +![info](../assets/images/oa-info.png) ## Multiple instances -To configure a second CDM database (e.g. for production), create it under its own name -and point the field's own flag at it: +Note that postgres integration tests in this library and others following the `oa-configurator` utility may perform destructive actions and therefore require their own test-enabled configuration. This is typically only required for development use-cases. Test database configuration is separate to the inclusion of multiple CDM database connections (e.g. for staging/production). To configure a second CDM database, create it under its own name and point the field's own flag at it: ```bash omop-config databases add cdm_db_prod --kind cdm --connection cdm_prod omop-config configure omop_alchemy --cdm-db cdm_db_prod ``` -This creates `cdm_db_prod` without touching the existing `cdm_db`. There is no "default" -toggle to flip afterward; each deployment's `configure` call names the entry it wants -directly. +This creates `cdm_db_prod` without touching the existing `cdm_db`. There is no "default" toggle to flip afterward; each deployment's `configure` call names the entry it wants directly. See the [oa-configurator integration guide](https://AustralianCancerDataNetwork.github.io/oa-configurator/integration/#multiple-environments) for the full multi-environment guide. diff --git a/docs/getting-started/index.md b/docs/getting-started/index.md index 0b3f91c..39c58f6 100644 --- a/docs/getting-started/index.md +++ b/docs/getting-started/index.md @@ -1,11 +1,8 @@ # Getting Started -This section introduces the core ideas behind OMOP Alchemy and how to begin using it in -your own projects. +This section introduces the core ideas behind OMOP Alchemy and how to begin using it in your own projects. -OMOP Alchemy is designed to be safe to import anywhere and flexible enough to integrate -with existing pipelines, research workflows, and analysis environments. These pages -cover installation, maintenance tooling, and a minimal quickstart to orient you. +These pages cover installation, maintenance tooling, and a minimal quickstart for orientation. --- diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 18800b3..d154196 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -87,28 +87,23 @@ This is supported for postgres only. Use engine_with_replica_role when: -* Creating / refreshing materialized views -* Running schema-level operations that might trigger independent sessions +* Running schema-level operations that may open independent sessions * Using tooling that opens its own connections ## Optional PostgreSQL full-text search -OMOP Alchemy can optionally integrate with PostgreSQL full-text search for selected -vocabulary text fields. +OMOP Alchemy can optionally integrate with PostgreSQL full-text search for selected vocabulary text fields. -This feature is **not required** to use the library and is intentionally treated as an -optional enhancement rather than part of the core OMOP schema. +This feature is **not required** to use the library and is an optional enhancement rather than part of the core OMOP schema. At the library level: - query helpers can fall back to inline `to_tsvector(...)` expressions -- optional sidecar `tsvector` columns can be registered into SQLAlchemy metadata when - they exist in the database +- optional sidecar `tsvector` columns can be registered into SQLAlchemy metadata when they exist in the database At the database level: -- PostgreSQL-only sidecar columns and optional GIN indexes can be managed through the - maintenance CLI +- PostgreSQL-only sidecar columns and optional GIN indexes can be managed through the maintenance CLI - those sidecar columns are populated explicitly rather than auto-generated Typical maintenance workflow: @@ -124,6 +119,8 @@ If you later reload vocabulary data, rerun: omop-alchemy fulltext populate ``` +![fulltext](../assets/images/oa-fulltext.png) + For the full design and query patterns, see: - [PostgreSQL Full-Text Search](../advanced/fulltext.md) diff --git a/docs/getting-started/maintenance.md b/docs/getting-started/maintenance.md index a58b64c..336b174 100644 --- a/docs/getting-started/maintenance.md +++ b/docs/getting-started/maintenance.md @@ -1,13 +1,6 @@ # Maintenance CLI -The `omop-alchemy` maintenance CLI handles everything you need to operate an OMOP CDM -database: creating tables, loading Athena vocabularies, managing indexes and foreign key -enforcement, running health checks, and taking backups. It talks directly to a SQLAlchemy -engine, so all connection details are controlled by the same engine URL configuration you -use for the ORM. - -> **Alpha status** -> Treat this CLI as alpha operational tooling. Interfaces and behavior may still change. +The `omop-alchemy` maintenance CLI handles everything you need to operate an OMOP CDM database: creating tables, loading Athena vocabularies, managing indexes and foreign key enforcement, running health checks, and taking backups. It talks directly to a SQLAlchemy engine, so all connection details are controlled by the same engine URL configuration you use for the ORM. --- @@ -19,8 +12,7 @@ Database connection and CDM schema come from [oa_configurator](../getting-starte ## Backend support -Some commands depend on PostgreSQL-specific features and will return a clear error -if you run them against SQLite. +Some commands depend on PostgreSQL-specific features and will return an error if you run them against an unsupported backend. | Command group | Requires PostgreSQL | Why | | --- | --- | --- | @@ -40,8 +32,7 @@ if you run them against SQLite. ### Fresh database setup -Use this when you are starting with an empty database and want to get an OMOP schema -populated from scratch. +Use this when you are starting with an empty database and want to get an OMOP schema populated from scratch. ```bash # 1. Create any OMOP tables that don't exist yet (safe to run on an existing DB) @@ -55,12 +46,9 @@ omop-alchemy load-vocab-source --athena-source ./athena_files omop-alchemy reset-sequences ``` -The `create-missing-tables` command compares ORM metadata against the live schema and -creates only what is missing. It is idempotent — running it again on a populated database -does nothing. +The `create-missing-tables` command compares ORM metadata against the live schema and creates only what is missing. It is idempotent — running it again on a populated database does nothing. -`load-vocab-source` automatically creates any missing vocabulary tables before loading, -so you can run it immediately after step 1 or even skip step 1 for vocabulary-only setups. +`load-vocab-source` automatically creates any missing vocabulary tables before loading, so you can run it immediately after step 1 or even skip step 1 for vocabulary-only setups. --- @@ -88,35 +76,22 @@ omop-alchemy fulltext populate ``` **About `--bulk-mode` (default on PostgreSQL):** -`load-vocab-source` disables FK triggers and drops vocabulary indexes once before the -load loop, then rebuilds them once at the end. This is much faster than the alternative -of toggling per table — for a full Athena export the difference can be 10–20×. SQLite -ignores this flag. Pass `--no-bulk-mode` if you need per-table rollback safety. +`load-vocab-source` disables FK triggers and drops vocabulary indexes once before the load loop, then rebuilds them once at the end. This is much faster than the alternative of toggling per table — for a full Athena export the difference can be 10–20×. SQLite ignores this flag. Pass `--no-bulk-mode` if you need per-table rollback safety. **About `--merge-strategy replace`:** -`replace` overwrites rows whose primary keys occur in the CSV; it does not delete rows -that are absent from the source. The explicit `truncate-tables` step above is therefore -required when the database must exactly mirror a new Athena export. Use `upsert` for -incremental vocabulary patches that must preserve existing conflicting rows. Use -`insert_if_empty` as the fastest path when the target tables are guaranteed empty. +`replace` overwrites rows whose primary keys occur in the CSV; it does not delete rows that are absent from the source. The explicit `truncate-tables` step above is therefore required when the database must exactly mirror a new Athena export. Use `upsert` for incremental vocabulary patches that must preserve existing conflicting rows. Use `insert_if_empty` as the fastest path when the target tables are guaranteed empty. **About `--quote-mode by_delimiter`:** -The default preserves double-quotes as data in tab-delimited Athena exports and uses -RFC-4180 quoting for comma-delimited files. Use `--quote-mode csv` only when the source -genuinely wraps fields in CSV quotes; `--quote-mode literal` forces quotes to remain data. -The `auto` mode samples content and is less predictable for large Athena files. +The default preserves double-quotes as data in tab-delimited Athena exports and uses RFC-4180 quoting for comma-delimited files. Use `--quote-mode csv` only when the source genuinely wraps fields in CSV quotes; `--quote-mode literal` forces quotes to remain data. The `auto` mode samples content and may be less predictable for large files. **About `--strict` on `foreign-keys enable`:** -`--strict` validates all FK relationships before re-enabling RI triggers. If violations -are found, no triggers are re-enabled and you get a report of the problematic rows. -Omit `--strict` to re-enable unconditionally. +`--strict` validates all FK relationships before re-enabling RI triggers. If violations are found, no triggers are re-enabled and you get a report of the problematic rows. Omit `--strict` to re-enable unconditionally. --- ### ETL bulk load cycle -Use this before and after a large clinical data load to avoid the overhead of FK and -index maintenance during insertion. +Use this before and after a large clinical data load to avoid the overhead of FK and index maintenance during insertion. ```bash # Before your ETL runs: suspend enforcement and remove indexes @@ -132,14 +107,9 @@ omop-alchemy foreign-keys enable --strict omop-alchemy analyze-tables --scope clinical ``` -`analyze-tables` refreshes planner statistics after a large load so query plans don't -degrade. `--scope clinical` targets only clinical tables; omit `--scope` to analyze -everything. +`analyze-tables` refreshes planner statistics after a large load so query plans don't degrade. `--scope clinical` targets only clinical tables; omit `--scope` to analyze everything. -`reset-sequences` ensures that any auto-increment columns are positioned above the -maximum key value present in the table. This matters when your ETL inserts explicit IDs -(common in OMOP) — without a reset, the next ORM insert would try to reuse an ID that -already exists. +`reset-sequences` ensures that any auto-increment columns are positioned above the maximum key value present in the table. This matters when your ETL inserts explicit IDs (common in OMOP) — without a reset, the next ORM insert would try to reuse an ID that already exists. --- @@ -151,8 +121,7 @@ already exists. omop-alchemy doctor ``` -Runs a fast, non-destructive pass over connection readiness, schema drift, and FK -trigger status. The output tells you what is wrong and what to do about it. +Runs a fast, non-destructive pass over connection readiness, schema drift, and FK trigger status. The output tells you what is wrong and what to do about it. **Deep FK validation (PostgreSQL):** @@ -160,9 +129,7 @@ trigger status. The output tells you what is wrong and what to do about it. omop-alchemy doctor --deep ``` -Adds a full FK constraint scan — it actually queries the data to find rows that violate -declared FK relationships. On large databases this can be slow; use it when you suspect -data integrity issues after an ETL or vocabulary patch. +Adds a full FK constraint scan — it actually queries the data to find rows that violate declared FK relationships. On large databases this can be slow; use it when you suspect data integrity issues after an ETL or vocabulary patch. **Full environment introspection:** @@ -170,10 +137,7 @@ data integrity issues after an ETL or vocabulary patch. omop-alchemy info ``` -Shows the active engine URL, installed backend driver, OMOP Alchemy version, optional -dependency state (orm-loader, psycopg2/psycopg, etc.), and which maintenance commands -are available given the current backend. Run this first when diagnosing "why doesn't -this command work". +Shows the active engine URL, installed backend driver, OMOP Alchemy version, optional dependency state (orm-loader, psycopg2/psycopg, etc.), and which maintenance commands are available given the current backend. Run this first when diagnosing "why doesn't this command work". **When doctor reports a problem:** @@ -188,8 +152,7 @@ this command work". ### Schema drift -The `reconcile-schema` command compares your ORM metadata against the live database and -reports what it finds: +The `reconcile-schema` command compares your ORM metadata against the live database and reports what it finds: ```bash omop-alchemy reconcile-schema @@ -203,15 +166,13 @@ Output categories: - **matched** — table exists in both and metadata is consistent. - **drifted** — table exists in both but column definitions differ (types, nullability, defaults). The CLI does not auto-migrate; you need to handle schema migrations manually. -For safe deployment: run `reconcile-schema` first, then `create-missing-tables --dry-run`, -then `create-missing-tables`. +For safe deployment: run `reconcile-schema` first, then `create-missing-tables --dry-run`, then `create-missing-tables`. --- ### Full-text search sidecars -Full-text search support adds `tsvector` sidecar columns (and `GIN` indexes) to the -`concept` and `concept_synonym` tables, enabling fast text search over vocabulary. +Full-text search support adds `tsvector` sidecar columns (and `GIN` indexes) to the `concept` and `concept_synonym` tables, enabling fast text search over vocabulary. ```bash # Install the sidecar columns and indexes (once, after vocabulary tables exist) @@ -221,8 +182,7 @@ omop-alchemy fulltext install omop-alchemy fulltext populate ``` -**You must rerun `fulltext populate` after every vocabulary reload.** Sidecar vectors -do not auto-refresh when the underlying concept data changes. +**You must rerun `fulltext populate` after every vocabulary reload.** Sidecar vectors do not auto-refresh when the underlying concept data changes. To remove the sidecars: @@ -230,19 +190,15 @@ To remove the sidecars: omop-alchemy fulltext drop ``` -The `--regconfig` option controls the PostgreSQL text search configuration -(default `english`). For multilingual vocabularies, use a suitable config such as -`simple`. +The `--regconfig` option controls the PostgreSQL text search configuration (default `english`). For multilingual vocabularies, use a suitable config such as `simple`. -For query-side usage and optional ORM metadata registration, see -[PostgreSQL Full-Text Search](../advanced/fulltext.md). +For query-side usage and optional ORM metadata registration, see [PostgreSQL Full-Text Search](../advanced/fulltext.md). --- ### Backup and restore -These commands wrap `pg_dump` and `pg_restore` / `psql`. PostgreSQL client tools must -be installed and on `PATH`. +These commands wrap `pg_dump` and `pg_restore` / `psql`. PostgreSQL client tools must be installed and on `PATH`. ```bash # Create a backup (custom format is recommended — smaller and restorable in parallel) @@ -267,8 +223,7 @@ omop-alchemy restore-database ./cdm-backup.dump \ - For `plain` format, the schema is embedded in the SQL dump; no selective schema restore is possible. - For `custom` format, `pg_restore` can be invoked manually with `-n ` for selective schema restore. -Use `--dry-run` on `backup-database` to see the `pg_dump` command that would be run -without executing it. +Use `--dry-run` on `backup-database` to see the `pg_dump` command that would be run without executing it. --- @@ -276,9 +231,7 @@ without executing it. ### Bulk load or vocabulary reload fails mid-way -If `load-vocab-source` (with `--bulk-mode`) or your ETL process fails after FK triggers -and indexes have been disabled, they stay disabled. The database continues to accept -writes but does not enforce FK constraints, and queries may use slow sequential scans. +If `load-vocab-source` (with `--bulk-mode`) or your ETL process fails after FK triggers and indexes have been disabled, they stay disabled. The database continues to accept writes but does not enforce FK constraints, and queries may use slow sequential scans. To recover: @@ -301,11 +254,9 @@ omop-alchemy foreign-keys enable --strict omop-alchemy foreign-keys validate ``` -This reports exactly which tables have violations, which constraints are affected, and -how many rows fail. Fix the data, then retry `foreign-keys enable --strict`. +This reports exactly which tables have violations, which constraints are affected, and how many rows fail. Fix the data, then retry `foreign-keys enable --strict`. -If you need to re-enable FK triggers despite the violations (for example, to allow the -application to run while you investigate), use `foreign-keys enable` without `--strict`. +If you need to re-enable FK triggers despite the violations (for example, to allow the application to run while you investigate), use `foreign-keys enable` without `--strict`. ### Sequences are out of sync after a bulk insert @@ -316,8 +267,7 @@ omop-alchemy reset-sequences # all managed tables omop-alchemy reset-sequences --vocab # vocabulary tables only ``` -`reset-sequences` sets each owned sequence to `MAX(pk) + 1`. It reports every table -it touches and the old/new sequence positions. +`reset-sequences` sets each owned sequence to `MAX(pk) + 1`. It reports every table it touches and the old/new sequence positions. --- diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md index 40f790f..7f24586 100644 --- a/docs/getting-started/quickstart.md +++ b/docs/getting-started/quickstart.md @@ -1,10 +1,5 @@ # Quickstart -`OMOP_Alchemy` itself makes no assumptions about how PostgreSQL is provisioned: any -reachable instance works, local or otherwise. Docker orchestration for the OMOP stack -is handled at the workspace root (compose files there bring up every package's -containers as peers), not by a per-package `docker-compose.yaml` in this repo. - ## Prerequisites - A running PostgreSQL instance (any version supported by `omop_alchemy`'s SQLAlchemy dialects) @@ -29,6 +24,8 @@ The test suite includes PostgreSQL-specific tests that skip automatically unless > schema on every run. `test_cdm_db` must point to a **dedicated, empty test database**, never > to a database that contains real data. The test suite enforces this: it fails loudly (not skips) if the > configured database is not marked `test_only = true` in your config. +> +> Refer to the CI/CD workflows at [cava-devops](https://github.com/AustralianCancerDataNetwork/cava-devops/blob/main/.github/workflows/build-test-postgres.yml) for more details on how integration test runs are typically orchestrated. **Step 1 — Register a test database connection:** diff --git a/docs/index.md b/docs/index.md index 4192d15..34cb88f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -89,4 +89,4 @@ concept.is_standard ### Status -OMOP Alchemy is currently beta. The core model surface is stabilising; feedback is welcome. \ No newline at end of file +OMOP Alchemy is beta. Toolkit APIs are experimental and carry no compatibility guarantees; feedback is welcome. diff --git a/docs/toolkit/analytics.md b/docs/toolkit/analytics.md index 9e0da51..d7c0f67 100644 --- a/docs/toolkit/analytics.md +++ b/docs/toolkit/analytics.md @@ -1,27 +1,140 @@ -# analytics +# Clinical analytics -Clinical-domain logic lives beside the concept sets and policies that give it meaning. -The domain-neutral `core` and `episodes` packages provide the underlying resolution and -traversal mechanisms. +Analytics packages combine domain-neutral retrieval with governed concept sets and clinical interpretation. This is where a procedure becomes evidence of radiotherapy, a series of measurements becomes a weight trajectory, or percentage weight loss becomes a severity grade. -## oncology +## Oncology | API | Capability | |---|---| | `OncologyEpisode` | Classifies episode purpose and modality; traverses events linked to the episode and its direct children. | -| `structural_modalities` / `concept_modalities` | Preserve every evidenced modality so mixed treatment and SACT classification disagreements remain visible. | +| `structural_modalities` / `concept_modalities` | Preserve every evidenced modality so mixed-treatment and SACT classification disagreements remain visible. | | `structural_modality` / `concept_modality` | Select one deterministic modality in radiotherapy, surgery, diagnostic/staging, SACT priority order. | | `OncologyProcedure` / `OncologyDrugExposure` | Add governed `is_radiotherapy`, `is_surgery`, `is_diagnostic_staging`, and `is_sact` questions to CDM facts. | | `RTDoseSummary.from_procedures(...)` | Constructs one radiotherapy summary; `summarize_rt_procedures_by(...)` groups before construction. | | `SACTDoseSummary.from_exposures(...)` | Constructs one SACT summary; `summarize_sact_exposures_by(...)` groups before construction. | | `OncologyEpisodeEvent` | Resolves oncology-aware facts while retaining episode-event diagnostics. | -Governed membership has two access modes: +`OncologyEpisode` is the main entry point for an episode-centred oncology analysis. Keep the object attached to its SQLAlchemy session while accessing properties that traverse related events or resolve vocabulary-backed concept groups: -| Access form | Behaviour | -|---|---| -| Loaded instance property | Expands once per vocabulary identity, then uses cached O(1) membership. Initial classification requires a live session; a cached result remains the snapshot computed while the instance was attached. | -| Class-level hybrid expression | Emits a database subquery for each query and does not use the Python expansion cache. | +```python +from sqlalchemy.orm import Session + +from omop_alchemy.toolkit.analytics.oncology import OncologyEpisode + +with Session(engine) as session: + episode = session.get(OncologyEpisode, episode_id) + if episode is None: + raise LookupError(f"Unknown episode: {episode_id}") + + treatment_episodes = episode.child_treatment_episodes + modalities = episode.structural_modalities + sact = episode.sact_dose_summaries_by_drug_concept + radiotherapy = episode.rt_dose_summaries_by_site +``` + +The episode includes events linked directly to it and events linked to its direct children. This supports a regimen whose drug exposures or procedures are recorded against cycle-level child episodes without flattening the episode hierarchy itself. + +```mermaid +flowchart TD + OE["OncologyEpisode"] --> Direct["Events linked directly to this episode"] + OE --> Children["child_treatment_episodes"] + Children --> ChildEvents["Events linked to those children
(e.g. cycle-level drug exposures)"] + Direct --> Pool["Evidence pool for
modalities, dose summaries, weight loss"] + ChildEvents --> Pool +``` + +### Modality evidence + +An episode can contain evidence for more than one treatment modality. `structural_modalities` and `concept_modalities` therefore return sets rather than forcing the record into a single label: + +- structural evidence treats any linked drug exposure as SACT evidence and uses governed concepts for radiotherapy, surgery, and diagnostic or staging procedures; +- concept evidence requires drug exposures to belong to the governed SACT concept set as well. + +Comparing the two sets makes source-structure and terminology disagreements visible: + +```python +structural = episode.structural_modalities +governed = episode.concept_modalities + +if structural != governed: + review_episode_modality(episode.episode_id, structural, governed) +``` + +When a caller needs one value, `structural_modality` and `concept_modality` apply a deterministic order: radiotherapy, surgery, diagnostic or staging, then SACT. This is a stable tie-break for mixed evidence, not a statement of clinical importance. Use the plural properties when mixed treatment matters to the analysis. + +```mermaid +flowchart LR + RT["Radiotherapy"] --> SUR["Surgery"] --> DX["Diagnostic / Staging"] --> SACT["SACT"] +``` + +The single-value properties return the first modality in this order for which the episode has evidence. + +### Treatment summaries + +`sact_exposures` contains linked exposures whose concepts belong to the governed SACT set. `sact_dose_summaries_by_drug_concept` groups them by drug concept; `sact_dose_summary` provides an all-SACT summary. The summary keeps source units and carries a `DoseEvaluability` result. Mixed units or missing quantities remain visible instead of being presented as a valid combined dose. + +`rt_procedures` applies the governed radiotherapy procedure set. Site-grouped and whole-episode summaries expose dates, procedure and modifier concepts, counts, quantities, and dose evaluability. OMOP Procedure Occurrence does not provide a universal radiotherapy dose model, so these summaries preserve the available evidence for a site-specific policy rather than inferring one. + +`OncologyProcedure` and `OncologyDrugExposure` expose the same governed classifications on individual facts. `OncologyEpisodeEvent` retains resolution diagnostics when a linked event cannot be loaded. + +### Condition modifiers and preferred stage + +The oncology package publishes lazy governed concept specifications for T, N, +M, and group stage, tumour grade, and metastatic disease. Laterality and tumour +size are exposed as governed scalar accessors. These declarations consume +`omop-semantics`; the narrow metastatic-disease descendant group requires +`omop-semantics` 0.6.1. Importing the module does not expand a vocabulary or +contact a database. + +Stage selection is a query policy over an already filtered canonical modifier +source enriched with `modifier_concept_code`. By default, pathological codes +(trimmed, case-insensitive codes beginning with `p`) rank before clinical codes +(beginning with `c`), with unclassified codes retained as fallback. Time and +canonical modifier identity then break ties deterministically. + +```python +from omop_alchemy.toolkit.analytics.oncology import preferred_stage_select + +# Earliest pathological, otherwise earliest clinical, otherwise unclassified. +preferred = preferred_stage_select( + stage_modifiers, + concept_code_column="modifier_concept_code", +) +``` + +The preference is immutable and query-scoped. Override it explicitly rather +than changing process-global state: + +```python +from omop_alchemy.toolkit.analytics.oncology import StageSelectionSpec +from omop_alchemy.toolkit.core.modifiers import ModifierSelectionPolicy + +clinical_first = preferred_stage_select( + stage_modifiers, + spec=StageSelectionSpec.clinical_first(), + concept_code_column="modifier_concept_code", +) + +chronological = preferred_stage_select( + stage_modifiers, + spec=StageSelectionSpec.chronological_only(), +) + +latest_pathological = preferred_stage_select( + stage_modifiers, + spec=StageSelectionSpec( + temporal_policy=ModifierSelectionPolicy.latest, + ), + concept_code_column="modifier_concept_code", +) +``` + +Basis-ranked selection requires an enriched source and an explicit +`concept_code_column=`. Chronological-only selection does not require the +enrichment. An empty basis priority disables pathological/clinical preference; +a non-empty priority must contain every `StageBasis` exactly once. + +::: omop_alchemy.toolkit.analytics.oncology.condition_modifiers ::: omop_alchemy.toolkit.analytics.oncology.OncologyEpisode options: @@ -60,16 +173,35 @@ Governed membership has two access modes: members: - from_exposures -## body_metrics +## Body metrics | API | Capability | |---|---| | `MeasurementReading.from_measurement(...)` | Reduces an OMOP measurement to the fields used by calculations and records its resolution source. | -| `MeasurementSeriesMixin` | Resolves normalized measurement series for an episode. | -| `WeightTrajectoryMixin` | Exposes normalized weight and height, BMI, BSA, windowed change, trajectories, and a dict-shaped typed summary. | +| `MeasurementSeriesMixin` | Resolves normalised measurement series for an episode. | +| `WeightTrajectoryMixin` | Exposes normalised weight and height, BMI, BSA, windowed change, trajectories, and a dict-shaped typed summary. | | `WeightChange` | Represents percentage change and whether it was evaluable; unevaluable change has `pct_change=None`. | | `WeightTrajectorySummary` | Types the DataFrame- and JSON-friendly mapping returned by `weight_trajectory_summary()`. | +`WeightTrajectoryMixin` turns an episode's weight measurements and the person's height measurements into a normalised longitudinal view. Weight is converted to kilograms, height to centimetres, and measurements with missing or unrecognised units are excluded from calculations. + +An episode that includes the mixin can produce a compact, tabular summary: + +```python +summary = episode.weight_trajectory_summary() + +print(summary["baseline_weight_kg"]) +print(summary["latest_weight_kg"]) +print(summary["pct_change_from_baseline"]) +print(summary["pct_change_from_baseline_evaluable"]) +``` + +The baseline is the first normalised weight in the resolved episode series and the latest weight is the last. Percentage change is negative for weight loss. A result separates its value from evaluability so that missing evidence is not confused with zero change. + +`pct_change_over(days)` compares the latest reading with the earliest reading inside the requested look-back period. `pct_change_trajectory()` returns every normalised point relative to baseline. `sustained_loss()` asks whether the final consecutive readings all meet a configurable loss threshold. These are deliberately distinct questions; choose the one that matches the analysis rather than treating them as interchangeable summaries of weight loss. + +Body-metric defaults resolve governed measurement and unit concepts. A deployment that uses local concepts can supply its own `BodyMetricRules` on the episode class. + ::: omop_alchemy.toolkit.analytics.body_metrics.MeasurementReading options: members: @@ -91,7 +223,7 @@ Governed membership has two access modes: - sustained_loss - weight_trajectory_summary -## adverse_events +## Adverse events | API | Policy | |---|---| @@ -99,4 +231,21 @@ Governed membership has two access modes: | `martin_weight_loss_grade(...)` | Applies the Martin et al. BMI-adjusted matrix. | | `critical_weight_loss_grade(...)` | Uses the Martin matrix when BMI is available and otherwise falls back to CTCAE-style bins. | +The adverse-event functions apply grading policy to an already calculated percentage change and, where available, BMI: + +```python +from omop_alchemy.toolkit.analytics.adverse_events import ( + critical_weight_loss_grade, +) + +grade = critical_weight_loss_grade( + pct_change=-8.2, + bmi=21.4, +) +``` + +`martin_weight_loss_grade()` applies the BMI-adjusted Martin et al. matrix. `ctcae_weight_loss_grade()` applies the CTCAE v5.0 physiological percentage-loss thresholds but does not infer intervention qualifiers such as hospitalisation, tube feeding, or parenteral nutrition. `critical_weight_loss_grade()` uses the Martin matrix when both percentage change and BMI are available and otherwise falls back to the percentage-only CTCAE-style grade. + +All three return `None` when the inputs needed by that policy are unavailable. They do not retrieve measurements or choose a baseline; those decisions remain in the body-metric layer. + ::: omop_alchemy.toolkit.analytics.adverse_events diff --git a/docs/toolkit/core.md b/docs/toolkit/core.md index 7df27de..faa7a94 100644 --- a/docs/toolkit/core.md +++ b/docs/toolkit/core.md @@ -1,14 +1,12 @@ -# core +# Core services -Foundational services with no clinical-domain assumptions. A concept resolver behaves -the same whether it is mapping tumour morphology or procedures; a patient timeline is -the same object whatever populates it. Domain-specific concept sets, thresholds, and -grading rules belong in [`analytics`](analytics.md), not here. +The core package handles problems that have the same meaning in every clinical domain: resolving a source term to an OMOP concept, identifying an event across CDM tables, arranging events on a timeline, and converting measurements to comparable units. -## Concept resolution +Generic database lifecycle operations do not belong in the clinical toolkit. Use [`orm-loader`](https://australiancancerdatanetwork.github.io/orm-loader/tables/mat_view/) to define, create, refresh, index, and drop materialised views. OMOP Alchemy owns the OMOP-specific query and row-grain decisions supplied to that infrastructure; a downstream application owns its registry, dependency policy, and deployment orchestration. See [Materialised views](materialized-views.md) for the integration boundary. -Turns a declarative description of *which* concepts belong in a lookup into a runtime -resolver that maps free text and source codes to OMOP concept IDs. +## Resolve source data to concepts + +Suppose an intake system supplies the text `Adenocarcinoma of lung` rather than an OMOP concept ID. A resolver limits the eligible vocabulary rows and applies the same text normalisation when it builds its lookup and when it handles an incoming value: ```python from omop_alchemy.toolkit.core.concepts import make_concept_resolver @@ -18,23 +16,193 @@ resolver = make_concept_resolver( name="condition lookup", domain_id="Condition", ) + concept_id = resolver.lookup("Adenocarcinoma of lung") ``` +Creating the resolver reads the vocabulary tables, so create it once for a mapping workflow and reuse it. `LookupSpec`, `LookupIndex`, and `ConceptResolver` expose the individual stages when you need to control indexed fields, normalisation, or resolver lifetime. `ConceptResolverRegistry` provides lazy construction and caching when an application maintains several lookups. + +Concept groups answer the complementary question: whether a known concept belongs to a governed set. A resolved group supports both in-memory membership and a SQLAlchemy expression derived from the same specification, so filtering loaded objects and filtering in SQL do not require separate definitions. + +Domain packages can declare module-level governed groups with `SemanticUnitRef(value_set, unit)`. The reference loads the optional `omop-semantics` runtime only when its parent, excluded-parent, or exact IDs are read. It always exposes a complete governed unit; narrower group definitions belong in `omop-semantics`, not in consumer-side adapters. + +Configuration-driven concept sets use `RuntimeConceptSetSpec`. It records exact and ancestral inclusions and exclusions without touching the database; see [Runtime concept sets](query-contracts.md#runtime-concept-sets) for the set semantics and current execution boundary. + +## Resolve concepts to standard concepts + +Most source concepts resolve to one standard concept. Some source concepts represent several clinical meanings, however, and OMOP maps those to several standard concepts. `standard_concept_mapping_select()` therefore returns one row per valid `Maps to` relationship rather than choosing one target: + +```python +from datetime import date + +from omop_alchemy.toolkit.core.concepts import ( + StandardConceptMappingSpec, + standard_concept_mapping_select, +) + +mapping_query = standard_concept_mapping_select( + StandardConceptMappingSpec( + source_concept_ids=(source_concept_id,), + valid_on=date(2026, 1, 1), + ) +) + +mapping_rows = session.execute(mapping_query).mappings().all() +``` + +Each row carries the source and standard concept identifiers, vocabularies, codes, and names alongside the relationship validity dates. Invalid relationships, invalid targets, non-standard targets, and other relationship types are excluded. A standard concept's `Maps to` self-map is returned normally. + +Supplying `valid_on` makes the relationship and target date ranges part of the query, which is useful when a result must be reproducible against a dated vocabulary release. The query does not follow replacement relationships or `Maps to value`: those relationships answer different questions and should be handled by purpose-specific queries when a toolkit consumer needs them. + ::: omop_alchemy.toolkit.core.concepts -## Patient timelines +## Identify events across CDM tables + +A Measurement and a Procedure Occurrence may have the same numeric primary key. Code that combines tables must therefore carry the source table as part of event identity: + +```python +from omop_alchemy.toolkit.core.events import ClinicalEventIdentity + +measurement = ClinicalEventIdentity("measurement", 7) +procedure = ClinicalEventIdentity("procedure_occurrence", 7) + +assert measurement != procedure +``` + +`ClinicalEventColumn` defines the common labels used when heterogeneous event tables are projected into one result. The required shape includes the person, table-scoped event identity, event date and datetime, clinical concept, and OMOP Field concept that identifies the source ID column. Optional labels cover numeric values, value concepts, and units. + +`ClinicalEventColumn.required_columns()` and `optional_columns()` expose these +groups as ordered enum tuples derived from the field-only row contracts. + +`canonical_event_union()` turns supported event models into that shared shape. Measurement and Observation retain numeric values, value concepts and units; Observation string values are outside this projection. Sources without those fields receive typed nulls so every branch of the union remains compatible: + +```python +from omop_alchemy.cdm.model import ( + Measurement, + Observation, + Procedure_Occurrence, +) +from omop_alchemy.toolkit.core.events import canonical_event_union + +events = canonical_event_union( + Measurement, + Observation, + Procedure_Occurrence, +) + +for event in session.execute(events).mappings(): + print(event["event_source_table"], event["event_id"], event["event_date"]) +``` + +The projection resolves its ID, clinical concept, date, source table, and Field concept through stable CDM event metadata shared with episode-event resolution. Bare `Measurement`, `Observation`, and `Device_Exposure` classes remain lightweight mappings for ETL, while their analytical views provide reference context, domain validation, and episode-event resolution. Importing analytics modules cannot change either the Core projection metadata or the default resolution target. `UnsupportedClinicalEventModelError` is raised before SQL execution when no supported CDM definition exists. + +The six clinical analytical Views inherit `ClinicalEventMixin`, which extends +`ModifierTargetMixin` with `has_complete_metadata()` and +`clinical_event_model_spec()`. The spec method can validate either the View's +columns or those of a supplied bare source model. The explicit registry still +controls event membership: `EpisodeView` is a modifier target rather than a +clinical event, and custom modifier sources may combine `ModifierSourceMixin` +with `ClinicalEventMixin` without registering as episode-event targets. -Projects a person's clinical rows — conditions, measurements, drug exposures — into a -single time-ordered event stream. This has its own dedicated page, since it predates the -rest of the toolkit reorg: see [Patient Timelines](../advanced/timelines.md). +```python +from omop_alchemy.cdm.model.clinical import Measurement, MeasurementView + +assert MeasurementView.has_complete_metadata() +spec = MeasurementView.clinical_event_model_spec(Measurement) +``` + +```mermaid +flowchart LR + M["Measurement
measurement_id"] --> U["canonical_event_union()"] + O["Observation
observation_id"] --> U + P["Procedure_Occurrence
procedure_occurrence_id"] --> U + U --> S["canonical shape
person_id · event_id · event_source_table
event_field_concept_id · event_date · event_concept_id
(+ value / value_concept / unit where supported)"] +``` + +Each source model contributes its own ID column and Field concept; branches without a value or unit receive typed nulls so the union stays one consistent shape regardless of which models are combined. + +The [query contracts](query-contracts.md) explain how the canonical shape participates in episode attachment. + +## Resolve OMOP modifiers + +Measurement and Observation both implement the OMOP polymorphic modifier link, +but use different physical column names. `canonical_modifier_projection()` and +`canonical_modifier_union()` normalize those tables to one shape containing a +table-scoped modifier identity, a Field-concept-scoped target identity, the +modifier date and concept, and all four OMOP value representations. + +`ModifierColumn.required_columns()` and `value_columns()` expose the ordered +modifier label groups without adding methods to the row Protocols. + +Supported source and target models are declared explicitly in immutable +metadata. Shared event fields and the six clinical target definitions are +derived from the clinical-event registry; Episode is the only target extension. +This keeps imports deterministic without maintaining a second copy of the CDM +Field concepts and native event columns. + +```python +from omop_alchemy.cdm.model import Measurement, Observation +from omop_alchemy.toolkit.core.modifiers import canonical_modifier_union + +modifiers = canonical_modifier_union(Measurement, Observation).subquery() +``` + +A numeric modifier ID is unique only inside its source table. Likewise, a +numeric target ID is meaningful only with `target_field_concept_id`. Preserve +both parts of both identities when a query is joined, ranked, or materialized. + +Use `modifier_target_queries()` before reducing repeated modifiers. The accepted +link must agree on target ID, target Field concept, and person. Supported targets +are Condition Occurrence, Measurement, Observation, Procedure Occurrence, Drug +Exposure, Device Exposure, and Episode. Episode is deliberately a modifier +target without being added to the clinical-event union. + +```python +from omop_alchemy.cdm.model import Condition_Occurrence, Measurement +from omop_alchemy.toolkit.core.modifiers import modifier_target_queries + +result = modifier_target_queries( + Measurement, + Condition_Occurrence, + diagnostics=True, +) + +valid_modifiers = result.matches +rejected_links = result.diagnostics +``` + +Diagnostics distinguish incomplete target identities, unsupported target Field +concepts, missing target events, and cross-person links. Missing-row diagnostics +for a caller-supplied filtered target projection are relative to that projection; +use a complete target model when absence from the CDM itself is the question. + +`selected_modifier_select()` then applies an explicit earliest/latest policy. +The default partition is `(person_id, target_field_concept_id, target_event_id)`; +date, non-null datetime, source table, and modifier ID form its deterministic +order. Rows without a complete target identity do not participate. + +::: omop_alchemy.toolkit.core.modifiers + +## Work with a patient timeline + +The timeline adapter presents conditions, measurements, observations, and drug exposures as a single ordered sequence while retaining each row's source identity and value semantics. Use it when an application needs to display or serialise a patient's chronology rather than build a set-based analytical query. + +The timeline has a dedicated guide with session requirements, event mappings, and extension points: [Patient timelines](../advanced/timelines.md). + +## Convert body measurements + +Body-size calculations require weights and heights to use consistent units. The default conversion rules use the unit concept recorded on each measurement: + +```python +from omop_alchemy.toolkit.core.units import default_body_unit_conversion_rules + +rules = default_body_unit_conversion_rules() +weight_kg = rules.normalize_weight_kg(180.0, rules.units.lb) +height_cm = rules.normalize_height_cm(70.0, rules.units.inch) +``` -## Unit conversion +An unknown unit, a missing unit, or a missing value produces `None`; values are never passed through as though they were already normalised. Deployments with local unit concepts can construct `BodyUnitConversionRules` with their own `BodySizeUnitConcepts` mapping. -Converts measurement values to canonical units. Kilograms, pounds, centimetres, and -inches mean the same thing in every clinical domain, so the conversion rules live here -rather than with any one domain's measurement logic — see -[`analytics.body_metrics`](analytics.md#body_metrics) for where those domain-specific -measurements are resolved and normalised using these rules. +Clinical choices built on those conversions, including which measurements constitute baseline weight and how change is graded, belong to [`analytics.body_metrics`](analytics.md#body-metrics) and [`analytics.adverse_events`](analytics.md#adverse-events). ::: omop_alchemy.toolkit.core.units diff --git a/docs/toolkit/episodes.md b/docs/toolkit/episodes.md index ef287af..20c71b1 100644 --- a/docs/toolkit/episodes.md +++ b/docs/toolkit/episodes.md @@ -1,72 +1,155 @@ -# episodes +# Episodes -Domain-neutral machinery for building episodes and retrieving what belongs to them. A -drug episode behaves the same whether the drug is a cytotoxic agent or an antibiotic, so -everything here takes concept filters and grouping keys as parameters rather than -assuming a clinical specialty. Domain-specific episode classes — for example -`OncologyEpisode` — compose these pieces with their own concept sets and live in -[`analytics`](analytics.md). +Episode APIs answer two related questions: how episodes relate to one another, and which clinical facts belong to an episode. They do not assume a specialty. Oncology-specific episode types compose these APIs with governed oncology concepts in the [analytics package](analytics.md#oncology). -## derivation +## Retrieve facts from an episode -How episodes are constructed and related to one another — building episode queries and -resolving parent/child hierarchy, written against the raw `Episode`/`Episode_Event` -tables rather than any materialised view. +For a treatment episode, a common first task is to retrieve its linked drug exposures and group them by drug concept: -Not yet populated. The equivalent built against materialised-view subclasses lives in -`omop-constructs`. - -## handling +```python +from omop_alchemy.cdm.model.structural import EpisodeView +from omop_alchemy.toolkit.episodes.handling import DrugEpisodeMixin -What is inside an episode once it exists. -**Linked drug exposures.** `DrugEpisodeMixin` adds retrieval and grouped summaries to -any episode view: +class TreatmentEpisode(DrugEpisodeMixin, EpisodeView): + _drug_concept_ids = treatment_drug_concept_ids -```python -from omop_alchemy.toolkit.episodes.handling import DrugEpisodeMixin -class MyEpisode(DrugEpisodeMixin, EpisodeView): - _drug_concept_ids = my_concept_ids +episode = session.get(TreatmentEpisode, episode_id) +if episode is None: + raise LookupError(f"Unknown episode: {episode_id}") -episode.drug_exposures # resolved Drug_Exposure rows -episode.drug_exposure_summaries_by() # grouped by drug concept by default +exposures = episode.drug_exposures +summaries = episode.drug_exposure_summaries_by() ``` -Construct a summary for an already selected set of rows through the summary -type itself: +`drug_exposures` uses explicit `Episode_Event` links by default. `_drug_concept_ids` limits the rows to the concepts meaningful for this episode type, and `drug_exposure_summaries_by()` groups the selected rows by `drug_concept_id`. Pass a key function when another grouping, such as ingredient or regimen member, is more useful. -```python -from omop_alchemy.toolkit.episodes.handling import DrugExposureSummary +If rows have already been selected elsewhere, construct or group summaries directly: -summary = DrugExposureSummary.from_exposures(exposures, group_key="regimen-a") +```python +from omop_alchemy.toolkit.episodes.handling import ( + DrugExposureSummary, + summarize_drug_exposures_by, +) + +regimen_summary = DrugExposureSummary.from_exposures( + regimen_exposures, + group_key="regimen-a", +) + +by_drug = summarize_drug_exposures_by( + regimen_exposures, + key=lambda exposure: exposure.drug_concept_id, +) ``` -Dose quantities are frequently not comparable across agents, because source units and -quantities arrive unnormalised. `DoseEvaluability` carries that judgement alongside the -number, so a summary that cannot be interpreted as a dose says so rather than presenting -a misleading total. +The generic summary reports counts, dates, source units, concepts, and a raw quantity total. A total is only clinically comparable when the source quantities have compatible meaning and units. Domain-specific summaries can attach `DoseEvaluability` to make that judgement explicit, as the oncology SACT and radiotherapy summaries do. + +## Explicit links and date windows + +An `Episode_Event` row is the strongest statement that a fact belongs to an episode, so linked facts are always retained. Some datasets do not populate these links consistently. A caller can opt into bounded date-window retrieval for drug exposures by setting `_include_window_drug_exposures = True` on its mixin class. + +Window retrieval must be paired with a meaningful concept filter. Without one, every same-person drug exposure inside the dates is eligible. The generic helper deliberately defaults to explicit links only. -**Explicit links versus admitted-by-window.** Facts linked through `Episode_Event` are -always honoured. `episode_attachment_window` computes the bounded, date-based fallback -window used when a caller opts in to admitting same-person facts that weren't explicitly -linked. +`episode_attachment_window()` provides a related bounded window for episode-attributable facts. Its lower bound is a configurable number of days before the episode start. Its upper bound is the recorded episode end, or a finite fallback after the start when the episode is open-ended. The finite fallback prevents an incomplete episode from absorbing the rest of a person's record. -**Resolution diagnostics.** `Episode_EventView.resolved_event` already resolves an -`Episode_Event` link best-effort, returning `None` on failure. `ResolvedEpisodeEvent` -extends it to explain *why* — a miscoded field concept, a target class not yet -registered, or a genuinely dangling reference: +## Understand unresolved episode links + +`Episode_EventView.resolved_event` returns the linked analytical ORM row when the field concept and target row can be resolved, otherwise `None`. Measurement, Observation, and Device Exposure links resolve to `MeasurementView`, `ObservationView`, and `Device_ExposureView`; their bare table mappings remain available for lightweight ETL. The target map uses the same stable CDM metadata as canonical event projections, so importing domain-specific analytics cannot change the default target. `ResolvedEpisodeEvent` preserves the resolution behaviour and adds a diagnostic that distinguishes three cases: + +- the field concept is not a recognised `ModifierFieldConcepts` value; +- the field concept is recognised but no ORM target class is registered for it; or +- the target row does not exist. ```python from omop_alchemy.toolkit.episodes.handling import ResolvedEpisodeEvent -ee = session.get(ResolvedEpisodeEvent, (episode_id, event_id, field_concept_id)) -ee.resolved_event # the resolved row, or None -ee.event_resolution_diagnostics # [] if resolved cleanly, otherwise why not +link = session.get( + ResolvedEpisodeEvent, + (episode_id, event_id, field_concept_id), +) + +if link is not None and link.resolved_event is None: + for diagnostic in link.event_resolution_diagnostics: + logger.warning("%s: %s", diagnostic.kind, diagnostic.message) ``` -Mix `ResolvedEpisodeEventMixin` into an episode view to reach diagnostics through -ordinary `episode.episode_events` traversal instead of a direct query — this is how -`OncologyEpisode` gets diagnostics on oncology-aware event resolution for free. +Use `ResolvedEpisodeEventMixin` on an episode view when diagnostics should be available through `episode.episode_events` rather than through a separate query. ::: omop_alchemy.toolkit.episodes.handling + +## Traverse an episode hierarchy + +`canonical_episode_projection()` selects the stable structural fields without joining vocabulary tables. Pass `include_concept_label=True` when a result intended for inspection or presentation also needs `episode_concept_name`. The vocabulary lookup is optional: an episode remains in the result with a null label when its concept is zero, absent, or unavailable in a subset vocabulary. + +```python +from omop_alchemy.toolkit.episodes.derivation import canonical_episode_projection + +episodes = canonical_episode_projection(include_concept_label=True) +``` + +For a parent episode, `episode_descendants()` returns a recursive CTE containing the root at depth zero and each descendant at its distance from that root: + +```python +from sqlalchemy import select + +from omop_alchemy.toolkit.episodes.derivation import episode_descendants + +hierarchy = episode_descendants(root_episode_id=episode_id) +statement = select( + hierarchy.c.episode_id, + hierarchy.c.episode_parent_id, + hierarchy.c.depth, +).order_by(hierarchy.c.depth, hierarchy.c.episode_id) + +rows = session.execute(statement).mappings().all() +``` + +Traversal follows parent IDs only within the same person and stops at a configurable maximum depth, which bounds malformed cyclic data. Set `include_root=False` when only descendants are needed. `direct_episode_relationship_projection()` provides a non-recursive parent-child result for callers that need one level only. + +`episode_event_hierarchy_projection()` joins the hierarchy to `Episode_Event` and retains the root episode, the episode that owns the link, and its depth. This lets a caller include child-linked evidence without encoding a specialty-specific number of child levels: + +```mermaid +flowchart TD + Root["Episode 1000 (root)
depth 0"] --> C1["Episode 1001
depth 1"] + Root --> C2["Episode 1002
depth 1"] + C1 --> GC1["Episode 1010
depth 2"] + C2 --> GC2["Episode 1011
depth 2"] + Ev(["Procedure_Occurrence 7
Episode_Event link"]) -. "joined via
episode_event_hierarchy_projection()" .-> GC1 +``` + +The link at depth 2 is visible to code operating on the root at depth 0, without the caller having to know how many levels separate them. + +## Describe episode attachment policy + +The derivation package provides declarative types for code that assigns events to episodes. The types keep four choices visible: whether explicit links take precedence, whether fallback may return one or several episodes, which side of an anchor date is preferred, and how candidates within that preference are ranked. + +For example, the following policy honours a valid explicit link and otherwise chooses one episode. Episodes that had started by the event date are considered before future episodes, and the nearest start date wins within that group: + +```python +from omop_alchemy.toolkit.episodes.derivation import ( + EpisodeAttachmentPolicy, + EpisodeWindowSpec, + TemporalRankingSpec, + TemporalSelectionPolicy, + TemporalSidePreference, +) + +attachment = EpisodeAttachmentPolicy.explicit_first_ranked +window = EpisodeWindowSpec( + days_prior=90, + open_end_fallback_days=365, +) +ranking = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + side_preference=TemporalSidePreference.on_or_before_anchor, +) +``` + +Pass the policy, window, and—only for ranked fallback—ranking to `episode_attachment_queries()` with a canonical event projection. The builder validates explicit links by event ID, Field-concept discriminator, episode ID, and person; suppresses fallback only after a valid link; and returns deterministic attachments plus optional diagnostics. All-in-window fallback accepts the window but rejects a ranking because it deliberately retains every admitted episode. `episode_window_predicate()`, `temporal_order_expressions()`, and `temporal_row_number()` remain available when a query needs the individual portable SQL pieces. Date arithmetic is implemented for PostgreSQL and SQLite; compiling it for another dialect fails rather than assuming PostgreSQL syntax. + +See [Query contracts](query-contracts.md) for the complete result shape, attachment example, boundaries, repeated-observation selection, and the distinction between absolute-nearest and already-started-first ranking. + +::: omop_alchemy.toolkit.episodes.derivation diff --git a/docs/toolkit/index.md b/docs/toolkit/index.md index 145568c..b5a1033 100644 --- a/docs/toolkit/index.md +++ b/docs/toolkit/index.md @@ -1,55 +1,47 @@ # Toolkit -`omop_alchemy.cdm` gives you the OMOP CDM schema as SQLAlchemy models. The toolkit is -what you build with them: vocabulary resolution, patient timelines, episode traversal, -domain analytics, and outbound export. - -!!! warning "Experimental" - The toolkit is newer and less battle-tested than the CDM models it sits on top of. - Module paths below an area (anything past `toolkit..`) may still move; - the area itself is the stable import surface. The CDM models themselves are not - affected by anything here. - -## Four tiers - -Each tier may depend only on the tiers before it in this list — `core` knows nothing of -episodes or clinical domains, and nothing depends on `integrations`. - -| Tier | Answers | Assumes a clinical domain? | -|---|---|---| -| [`core`](core.md) | Resolve concepts, build a patient timeline, convert units | No | -| [`episodes`](episodes.md) | Build episodes, retrieve what belongs to one | No | -| [`analytics`](analytics.md) | What does this value mean clinically? | Yes, one subpackage per domain | -| [`integrations`](integrations.md) | Export to an external data standard | No | - -## A worked example - -Querying an oncology episode pulls together all four tiers without you having to think -about the seams between them: concept resolution and timelines from `core`, drug -retrieval from `episodes`, oncology classification from `analytics`. +The toolkit turns OMOP rows into objects that answer clinical questions. For example, given the ID of an oncology episode, you can inspect its treatment modality, linked drug exposures, radiotherapy dose, and weight-loss assessment while the episode remains attached to a SQLAlchemy session: ```python from sqlalchemy.orm import Session + from omop_alchemy.toolkit.analytics.oncology import OncologyEpisode with Session(engine) as session: episode = session.get(OncologyEpisode, episode_id) + if episode is None: + raise LookupError(f"Unknown episode: {episode_id}") - episode.structural_modality # OncologyModality.SACT, .RADIOTHERAPY, ... - episode.drug_exposures # linked Drug_Exposure rows - episode.rt_dose_summary # RTDoseSummary, if any RT was given - episode.critical_weight_loss_grade # graded against Martin/CTCAE criteria + modalities = episode.structural_modalities + drug_exposures = episode.sact_exposures + radiotherapy = episode.rt_dose_summary + weight_loss = episode.critical_weight_loss_summary() ``` -## Import surface +This is still ordinary SQLAlchemy. `OncologyEpisode` is mapped to the OMOP episode view, and properties may load related rows or resolve governed vocabulary sets through the active session. The toolkit adds interpretation and reusable retrieval rules; it does not replace the CDM models or hide when database access is required. + +!!! warning "Experimental toolkit" + Toolkit APIs are experimental and may change without compatibility guarantees. Pin the package version in deployed applications and review release notes before upgrading. + +## Where to begin + +Choose the part of the toolkit that matches the question you are asking: + +| If you need to… | Start with | +|---|---| +| Resolve incoming text or source codes to OMOP concepts, compare measurements in common units, or represent events from several CDM tables consistently | [`core`](core.md) | +| Traverse episode relationships, retrieve episode-linked facts, or state how an event should be attached to an episode | [`episodes`](episodes.md) | +| Apply a clinical interpretation such as oncology modality, dose summarisation, body-metric analysis, or weight-loss grading | [`analytics`](analytics.md) | + +The dependency direction follows the same order. `episodes` can use `core`; `analytics` can use both. Lower layers never import a clinical specialty or an export format. This keeps general concepts such as event identity and unit conversion independent of the analyses that use them. + +## Public imports -Import from the area subpackage — `omop_alchemy.toolkit..` — not from a -specific module beneath it: +Import from the documented area package rather than from a file beneath it: ```python from omop_alchemy.toolkit.core.concepts import make_concept_resolver from omop_alchemy.toolkit.analytics.oncology import OncologyEpisode ``` -Each area's `__init__.py` re-exports its public names, and that's the part of the path -that stays stable. Files beneath it are free to move. +The area packages re-export their public API. Module names below those packages are implementation details. diff --git a/docs/toolkit/integrations.md b/docs/toolkit/integrations.md deleted file mode 100644 index 84605d2..0000000 --- a/docs/toolkit/integrations.md +++ /dev/null @@ -1,14 +0,0 @@ -# integrations - -Export to external data standards. Integrations sit at the outer edge of the toolkit — -one may use anything in `core`, `episodes`, or `analytics`, and nothing in those tiers -depends on an integration, so adding or changing an export format cannot affect the -clinical logic beneath it. Each integration brings its own heavyweight dependencies and -is gated behind an optional extra, so installing omop-alchemy does not pull in formats -you are not exporting to. - -## meds_standard - -Export to the [Medical Event Data Standard](https://github.com/Medical-Event-Data-Standard/meds). - -Not yet populated. diff --git a/docs/toolkit/materialized-views.md b/docs/toolkit/materialized-views.md new file mode 100644 index 0000000..f2c828a --- /dev/null +++ b/docs/toolkit/materialized-views.md @@ -0,0 +1,135 @@ +# Materialized views + +A materialized view is a persisted read model: an expensive or carefully defined query is computed once and then read like a table. This is useful when the same analytical shape is consumed repeatedly, but it also introduces a deployment lifecycle that ordinary query construction does not have. + +## The deployment contract + +The supported deployment contract is PostgreSQL with unqualified materialized-view names resolving to the `public` schema through the connection's `search_path`. Leave `schema` at its default when calling the lifecycle methods. Qualified non-`public` schemas and schema translation are not part of this package's materialized-view contract. + +OMOP Alchemy owns the OMOP models and the query-building vocabulary. [`orm-loader`](https://australiancancerdatanetwork.github.io/orm-loader/tables/mat_view/) owns the generic materialized-view definition and database lifecycle. Keeping that boundary means this package can describe an OMOP read model without growing a second implementation of DDL, refresh, index creation, or dependency ordering. + +This page explains how an OMOP Alchemy query becomes a managed read model. The linked [`orm-loader` materialized-view guide](https://australiancancerdatanetwork.github.io/orm-loader/tables/mat_view/) is the authoritative reference for the complete API, supported options, backend behavior, and generated reference documentation. + +The important design decision is the shape of the result. Give the view a stable name, build its contents with the same SQLAlchemy expressions used elsewhere in the toolkit, and make the row grain explicit in the query. The example below creates one row per person and measurement concept, which makes the unique index meaningful as well as useful for concurrent refresh. + +```python +import sqlalchemy as sa + +from omop_alchemy.cdm.model import Measurement +from orm_loader.mappers import MaterializedViewIndex +from orm_loader.mappers import MaterializedViewMixin + + +measurement_summary = ( + sa.select( + Measurement.person_id, + Measurement.measurement_concept_id.label("concept_id"), + sa.func.max(Measurement.measurement_date).label("last_measurement_date"), + sa.func.count().label("measurement_count"), + ) + .group_by(Measurement.person_id, Measurement.measurement_concept_id) +) + + +class MeasurementSummaryMV(MaterializedViewMixin): + __mv_name__ = "measurement_summary" + __mv_select__ = measurement_summary + __mv_indexes__ = ( + MaterializedViewIndex( + name="measurement_summary_identity_uq", + columns=("person_id", "concept_id"), + unique=True, + ), + ) +``` + +`orm-loader` does not infer or validate the logical grain of a selectable. Treat the query's grouping and joins as the source of truth, test that grain against representative data, and declare a matching unique index when the view needs concurrent refresh. The index is the database-level guarantee; a comment or a convention in the Python class is not. + +`__mv_dependencies__` is for dependencies between managed materialized views. Its values are matched against the names of the view classes passed to the refresh-order resolver, so a base OMOP table such as `measurement` does not create a lifecycle dependency by itself. Add a dependency when one materialized view reads another: + +```python +class PersonMeasurementSummaryMV(MaterializedViewMixin): + __mv_name__ = "person_measurement_summary" + __mv_select__ = sa.select( + sa.table( + "measurement_summary", + sa.column("person_id", sa.Integer), + sa.column("concept_id", sa.Integer), + sa.column("last_measurement_date", sa.Date), + sa.column("measurement_count", sa.Integer), + ) + ) + __mv_dependencies__ = {"measurement_summary"} +``` + +Use the class as a schema-level definition, not as a promise that OMOP Alchemy will map the resulting relation or schedule its deployment. If application code needs ORM-style reads, map the relation separately according to the application's identity and session requirements. + +## Treat creation as deployment + +Creation is usually part of an application migration, bootstrap command, or release step. Keep it explicit and make the existing-target policy deliberate: + +```python +with engine.begin() as connection: + MeasurementSummaryMV.create_mv(connection) +``` + +The default is idempotent: `create_mv()` emits `IF NOT EXISTS`, so an already-present view is left in place. That is convenient for repeatable bootstrap, but it does not detect definition drift. Pass `if_not_exists=False` when an existing target should fail the deployment and force an explicit replacement decision. + +The view's declared indexes are created by `create_mv()` by default. Pass `create_indexes=False` only when index creation is intentionally managed elsewhere. With an `Engine`, `orm-loader` manages the transaction for each backend operation; with a `Connection`, the caller controls the transaction and can keep view and index creation inside a larger migration transaction. If index creation fails after the view has been created through an engine, treat the deployment as incomplete and reconcile it before retrying. + +`with_data=False` is useful when the relation must exist before its source data is ready, but the resulting view cannot be queried until it has been refreshed: + +```python +MeasurementSummaryMV.create_mv(engine, with_data=False, if_not_exists=False) +# Load or migrate the source data, then make the read model available. +MeasurementSummaryMV.refresh_mv(engine) +``` + +For replacement or teardown, use the lifecycle options documented by [`orm-loader`](https://australiancancerdatanetwork.github.io/orm-loader/tables/mat_view/) rather than issuing raw `CREATE`, `DROP`, or `REFRESH` statements in this package. In particular, `cascade=True` is an explicit decision to remove dependent database objects as part of a drop. + +## Choose a refresh policy + +An ordinary refresh is the simple operational default: + +```python +MeasurementSummaryMV.refresh_mv(engine) +``` + +Concurrent refresh is a PostgreSQL feature for keeping the existing materialized view available while its contents are rebuilt. It requires an eligible unique index over the view, with no predicate or expression-based shortcut. `orm-loader` first fails closed when the class declares no unique index and then translates PostgreSQL's rejection when the live database still does not satisfy the requirement. Both paths raise `ConcurrentRefreshNotEligibleError`, with the database exception preserved as the cause when PostgreSQL produced one. + +```python +from orm_loader.backends import ConcurrentRefreshNotEligibleError + + +try: + MeasurementSummaryMV.refresh_mv(engine, concurrently=True) +except ConcurrentRefreshNotEligibleError as error: + logger.warning("Concurrent refresh unavailable: %s", error) + MeasurementSummaryMV.refresh_mv(engine) +``` + +Use the fallback only if serving a briefly stale or synchronously refreshed read model is acceptable. A failed statement can leave a caller-managed PostgreSQL transaction aborted, so roll that transaction back before issuing more statements. An `engine.begin()` context rolls back automatically when an exception leaves the context. + +## Orchestrate a family of views + +The application owns the registry because it knows which views belong to a deployment and which policy should govern them. `orm-loader` supplies the small amount of generic machinery needed to order managed views and invoke their lifecycle methods. + +```python +from orm_loader.mappers import ( + refresh_all_mvs, + resolve_mv_refresh_order, +) + + +ALL_MVS = [ + MeasurementSummaryMV, + PersonMeasurementSummaryMV, +] + +# Ordinary refreshes in dependency order. +refresh_all_mvs(engine, ALL_MVS) + +# For concurrent refreshes, retain the same order while choosing the policy. +for view_cls in resolve_mv_refresh_order(ALL_MVS): + view_cls.refresh_mv(engine, concurrently=True) +``` diff --git a/docs/toolkit/query-contracts.md b/docs/toolkit/query-contracts.md new file mode 100644 index 0000000..77d44d7 --- /dev/null +++ b/docs/toolkit/query-contracts.md @@ -0,0 +1,408 @@ +# Query contracts + +Consider a procedure recorded on the same day as two overlapping treatment episodes. The procedure may already have a valid `Episode_Event` link, or it may need to be assigned from dates alone. A reliable query has to answer several questions explicitly: what identifies the procedure, whether an explicit link takes precedence, whether fallback may attach it to one or both episodes, and how equally plausible candidates are ordered. + +The contracts on this page provide a common vocabulary for those decisions. They are small, immutable values that can be shared by query-building code, configuration, and tests. Projection and predicate helpers translate the contracts into SQLAlchemy statements without opening a database connection. + +!!! note "Construction and execution" + Creating a contract, statement, CTE, or predicate is side-effect free. Database access begins only when the resulting statement is executed through a connection or session. The toolkit supplies projection, attachment, hierarchy, temporal, concept-set, mapping, and observation builders. + +## Start with event identity + +OMOP primary keys are scoped to their source tables. These three rows represent three different events even though they all use event ID 7: + +| Source table | Event ID | Person | Date | +|---|---:|---:|---| +| Measurement | 7 | 101 | 20 January 2026 | +| Procedure Occurrence | 7 | 101 | 20 January 2026 | +| Observation | 7 | 202 | 20 January 2026 | + +`ClinicalEventIdentity` keeps the table and numeric ID together: + +```python +from omop_alchemy.toolkit.core.events import ClinicalEventIdentity + +measurement = ClinicalEventIdentity("measurement", 7) +procedure = ClinicalEventIdentity("procedure_occurrence", 7) +observation = ClinicalEventIdentity("observation", 7) + +assert len({measurement, procedure, observation}) == 3 +``` + +A cross-table projection needs more than an identity. `ClinicalEventColumn.required_columns()` defines the labels a consumer can rely on: + +| Column | Meaning | +|---|---| +| `person_id` | Person who owns the source event | +| `event_id` | Primary key in the source table | +| `event_source_table` | Table that scopes `event_id` | +| `event_field_concept_id` | OMOP Field concept naming the source table's ID column | +| `event_date` | Date used for temporal selection | +| `event_datetime` | Source datetime when one is available | +| `event_concept_id` | Primary clinical concept carried by the event | + +Numeric value, value concept, and unit labels are available through `ClinicalEventColumn.optional_columns()` when a source table supports them. + +The Field concept is not interchangeable with the event's clinical concept. For example, a Procedure Occurrence projection uses the Field concept for `procedure_occurrence.procedure_occurrence_id` as its discriminator and the row's `procedure_concept_id` as its clinical concept. + +Build one shared event stream by passing the source models to `canonical_event_union()`: + +```python +from omop_alchemy.cdm.model import Measurement, Observation, Procedure_Occurrence +from omop_alchemy.toolkit.core.events import canonical_event_union + +events = canonical_event_union( + Measurement, + Observation, + Procedure_Occurrence, +).subquery("clinical_events") +``` + +All branches expose the same labels. The source table and Field concept are literals derived from model metadata, so they remain available after the tables are combined. + +## Attach an event to an episode + +A complete attachment key adds the episode ID to the table-scoped event identity: + +```python +from omop_alchemy.toolkit.episodes.derivation import EpisodeAttachmentIdentity + +attachment = EpisodeAttachmentIdentity.from_event( + procedure, + episode_id=1002, +) + +assert attachment.event == procedure +``` + +Before accepting an explicit link, the query confirms that the event ID and Field concept identify a row in the supplied projection and that the event and episode belong to the same person. A link carrying another table's Field concept is outside that projection's scope. This is important because event IDs are unique only within their OMOP table: a Procedure Occurrence 7 link says nothing about Measurement 7, even when both numbers happen to be present. + +Attachment diagnostics therefore do not infer a discriminator error or missing target from non-matching rows. An arbitrary event projection may intentionally be filtered, so absence from it does not prove anything about the underlying OMOP table. `ResolvedEpisodeEvent.event_resolution_diagnostics` checks the target table named by the Field concept and reports unsupported fields or `dangling_event` when the row truly does not exist. A link that names an existing row but is clinically incorrect cannot be identified from the linkage columns alone. + +Once valid, an explicit link takes precedence under either explicit-first policy. Suppose Procedure Occurrence 7 is linked to episode 1002, while its date also falls inside the windows of episodes 1001 and 1002. The result is only `(procedure_occurrence, 7, 1002)`: fallback must not add episode 1001 or duplicate episode 1002. + +`EpisodeAttachmentPolicy` controls what happens when no valid explicit link exists: + +| Policy | Fallback behaviour | +|---|---| +| `explicit_only` | Leave the event unattached | +| `explicit_first_ranked` | Select one date-eligible episode using a separate ranking specification | +| `explicit_first_all_in_window` | Retain every date-eligible episode | + +```mermaid +flowchart TD + Start["Event from canonical projection"] --> Valid{"Valid explicit
Episode_Event link?"} + Valid -- "yes" --> Explicit["Attach via the explicit link
(precedence; fallback is not applied)"] + Valid -- "no" --> Policy{"EpisodeAttachmentPolicy"} + Policy -- "explicit_only" --> Unattached["Leave unattached"] + Policy -- "explicit_first_ranked" --> Ranked["Rank date-eligible episodes
(TemporalRankingSpec)"] --> One["Attach to one episode"] + Policy -- "explicit_first_all_in_window" --> Window["Every date-eligible episode
in the window"] --> Many["Attach to each eligible episode"] +``` + +Choosing between ranked and all-in-window fallback is a statement about result grain. Ranked fallback produces at most one episode per event. All-in-window fallback intentionally allows one event to appear against several overlapping episodes. + +`episode_attachment_queries()` applies the complete precedence rule. It accepts a canonical event statement or one supported event model, validates explicit links against `Episode_Event`, and applies fallback only to events that have no valid explicit link: + +```python +from omop_alchemy.toolkit.episodes.derivation import ( + EpisodeAttachmentPolicy, + EpisodeAttachmentDiagnostic, + TemporalRankingSpec, + TemporalSelectionPolicy, + episode_attachment_queries, +) + +attachment_queries = episode_attachment_queries( + events, + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ranking=TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + ), + include_diagnostics=True, +) + +attachments = session.execute(attachment_queries.attachments).mappings().all() +assert attachment_queries.diagnostics is not None +diagnostics = session.execute(attachment_queries.diagnostics).mappings().all() +typed_diagnostics = [ + EpisodeAttachmentDiagnostic.from_mapping(row) + for row in diagnostics +] +``` + +The attachment result preserves the event projection and adds `episode_id` and `attachment_method`. Its uniqueness key is `(event_source_table, event_id, episode_id)`. A valid explicit link may legitimately connect an event to more than one episode; each relationship remains a separate attachment under that key. + +Diagnostics are advisory rows and do not change the attachments. They identify cross-person links, fallback ambiguity, and events for which no valid explicit link or fallback candidate exists. `EpisodeAttachmentDiagnostic.from_mapping()` converts a raw SQLAlchemy mapping into a typed value carrying the event identity, projected and linked Field concepts, episode, candidate count, and message. + +## Rank fallback candidates + +Ranking has two independent parts: which side of the anchor date should be considered first, and how candidates on that side should be ordered. Keeping them separate supports both symmetric nearest-date matching and the common preference for an episode that had already started when the event occurred. + +For an event on 20 January, consider these episode starts: + +| Episode | Start date | Absolute distance | State on 20 January | +|---|---|---:|---| +| 1001 | 15 January | 5 days | Already started | +| 1003 | 21 January | 1 day | Not yet started | + +A side-neutral nearest policy selects episode 1003: + +```python +from omop_alchemy.toolkit.episodes.derivation import ( + TemporalRankingSpec, + TemporalSelectionPolicy, +) + +absolute_nearest = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", +) +``` + +If the analysis should prefer an episode that was underway when the event happened, apply a side preference before distance: + +```python +from omop_alchemy.toolkit.episodes.derivation import TemporalSidePreference + +already_started_first = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + side_preference=TemporalSidePreference.on_or_before_anchor, +) +``` + +```mermaid +flowchart TD + A["Event on 20 January"] --> N{"policy = nearest"} + A --> S{"policy = nearest,
side_preference =
on_or_before_anchor"} + N -->|"closest by absolute distance"| E1003["Episode 1003
starts 21 Jan · 1 day away"] + S -->|"already-started side considered first"| E1001["Episode 1001
starts 15 Jan · 5 days away"] +``` + +This policy selects episode 1001. Absolute distance still orders episodes within the preferred side; it simply does not allow a closer future episode to outrank every episode that had already started. `on_or_after_anchor` expresses the corresponding future-first rule. + +Apply the policy to SQLAlchemy columns with `temporal_order_expressions()`: + +```python +from omop_alchemy.cdm.model.structural import Episode +from omop_alchemy.toolkit.episodes.derivation import temporal_order_expressions + +ordering = temporal_order_expressions( + Episode.episode_start_date, + events.c.event_date, + Episode.episode_id, + already_started_first, +) + +candidate_episodes = candidate_episodes.order_by(*ordering) +``` + +`earliest` and `latest` are available when chronological position, rather than distance from the anchor, defines the result. Every policy ends with the named stable ID column in ascending order. If episodes 1001 and 1002 are otherwise tied, 1001 wins consistently rather than relying on database return order. + +### Date boundaries + +Lower and upper bounds are inclusive by default and can be changed independently through `EpisodeWindowSpec`. For an episode starting 15 January 2026 with a 90-day prior window, 17 October 2025 lies exactly on the lower boundary and is included under the default. If the episode ends on 5 February, that date is included while 6 February is not. + +Window admission and candidate ranking are separate decisions. `EpisodeWindowSpec` owns the finite interval and its open or closed boundaries. `TemporalRankingSpec` is accepted only by ranked fallback and describes how already-admitted candidates are ordered. + +`episode_window_predicate()` uses the same finite defaults as the in-memory episode window. It honours a recorded episode end and substitutes a bounded post-start end only when the end is missing: + +```python +from omop_alchemy.toolkit.episodes.derivation import ( + EpisodeWindowSpec, + episode_window_predicate, +) + +window = EpisodeWindowSpec( + days_prior=90, + open_end_fallback_days=365, + include_lower_bound=True, + include_upper_bound=True, +) + +inside_episode_window = episode_window_predicate( + events.c.event_date, + Episode.episode_start_date, + Episode.episode_end_date, + window=window, +) +``` + +## Select one repeated observation + +Repeated observations need the same explicit treatment of direction, grouping, and ties. For an anchor date of 20 January, suppose a person has these rows: + +| Observation ID | Date | Value | +|---:|---|---| +| 21 | 1 January | earlier | +| 22 | 20 January | anchor-a | +| 23 | 20 January | anchor-b | +| 24 | 21 January | after-anchor | + +The following specification chooses the latest observation on or before the anchor, grouping rows by person and observation concept: + +```python +from omop_alchemy.toolkit.episodes.derivation import ( + ObservationSelectionPolicy, + ObservationSelectionSpec, +) + +selection = ObservationSelectionSpec( + policy=ObservationSelectionPolicy.latest_on_or_before_anchor, + partition_by=("person_id", "observation_concept_id"), + stable_id_column="observation_id", + include_anchor_date=True, +) +``` + +Use `ranked_observation_select()` to apply the anchor filter before calculating row numbers: + +```python +from datetime import date + +from sqlalchemy import literal, select + +from omop_alchemy.cdm.model import Observation +from omop_alchemy.toolkit.episodes.derivation import ranked_observation_select + +ranked = ranked_observation_select( + Observation.__table__, + selection, + anchor_date=literal(date(2026, 1, 20)), +).subquery("ranked_observations") + +selected = select(ranked).where(ranked.c.observation_rank == 1) +``` + +Observation 24 is after the anchor and is therefore excluded. Observations 22 and 23 tie on date, so the stable ID selects 22. That tie-break creates reproducible output; it does not claim that one same-day clinical value is more correct. If every same-day value is meaningful, retain them by choosing a result grain that includes the observation ID instead of reducing the group to one row. + +Add `episode_id` or another field to `partition_by` when selection must occur separately within those groups. The partition is part of the clinical meaning of the result, not merely an optimisation detail. + +## Runtime concept sets + +Applications often receive concept selection as configuration rather than as a compile-time governed unit. `RuntimeConceptSetSpec` records four inputs: exact concepts and descendants to include, and exact concepts and descendants to exclude. + +```python +from omop_alchemy.toolkit.core.concepts import RuntimeConceptSetSpec + +concepts = RuntimeConceptSetSpec( + include_ancestor_ids=(100,), + include_exact_ids=(900,), + exclude_ancestor_ids=(400,), + exclude_exact_ids=(901,), + require_standard=True, + include_classification=False, +) +``` + +The intended set is: + +```text +(descendants of 100 OR exact concept 900) +AND NOT (descendants of 400 OR exact concept 901) +``` + +Exclusion wins when a concept is reached from both sides. With no inclusion, the set matches nothing. IDs are sorted and deduplicated when the specification is created. + +`require_standard` and `include_classification` apply while expanding ancestor descendants. Exact IDs are explicit configuration and are not removed if the deployed vocabulary is temporarily out of step with the configuration source, including after a concept has been de-standardised. The specification does not decide whether an exact numeric ID is present, active, standard, or classification in a particular vocabulary; validate and report those expectations at the configuration boundary without changing membership silently. + +`runtime_concept_predicate()` translates the specification into database-side `concept_ancestor` and `concept` predicates: + +```python +from sqlalchemy import select + +from omop_alchemy.cdm.model import Procedure_Occurrence +from omop_alchemy.toolkit.core.concepts import runtime_concept_predicate + +matching_procedures = select(Procedure_Occurrence).where( + runtime_concept_predicate( + Procedure_Occurrence.procedure_concept_id, + concepts, + ) +) +``` + +Constructing the specification or predicate performs no hierarchy expansion and no database access. Descendants are resolved by the database when the surrounding statement is executed. Exact IDs are rendered as parameters, so very large externally supplied lists should be staged as rows and joined rather than pushed through a single `IN` predicate. + +Some applications compose positive and negative rules independently rather than collecting them into one runtime set. `descendant_concept_select()` provides the lower-level hierarchy operation for that case and returns each matching descendant once: + +```python +from omop_alchemy.toolkit.core.concepts import descendant_concept_select + +matching_procedures = select(Procedure_Occurrence).where( + Procedure_Occurrence.procedure_concept_id.in_( + descendant_concept_select((100, 200)) + ), + Procedure_Occurrence.procedure_concept_id.not_in( + descendant_concept_select((400,)) + ), +) +``` + +Use `RuntimeConceptSetSpec` when the inclusions and exclusions form one configured set with exclusion precedence. Use `descendant_concept_select()` when the surrounding query or rule model owns how separate predicates are combined. + +## Canonical modifier queries + +`ModifierIdentity(modifier_source_table, modifier_id)` and +`ModifierTargetIdentity(target_field_concept_id, target_event_id)` make both +table scopes explicit. The canonical source columns are: + +| Role | Columns | +|---|---| +| Source identity | `modifier_source_table`, `modifier_id` | +| Target identity | `target_field_concept_id`, `target_event_id` | +| Clinical row | `person_id`, `modifier_date`, `modifier_datetime`, `modifier_concept_id` | +| Nullable values | `value_as_number`, `value_as_concept_id`, `unit_concept_id`, `value_as_string` | + +Target validation is intentionally performed before selection. A valid link +matches target Field concept, target ID, and person. This prevents a modifier +for Condition Occurrence 7 from competing with one for Procedure Occurrence 7, +and prevents a malformed cross-person link from replacing valid evidence. + +```python +from omop_alchemy.cdm.model import Condition_Occurrence, Measurement +from omop_alchemy.toolkit.core.modifiers import ( + ModifierSelectionPolicy, + ModifierSelectionSpec, + modifier_target_queries, + selected_modifier_select, +) + +resolved = modifier_target_queries(Measurement, Condition_Occurrence) +selected = selected_modifier_select( + resolved.matches, + spec=ModifierSelectionSpec(policy=ModifierSelectionPolicy.earliest), +) +``` + +The default selection partition includes person and both target identity columns. If one input contains several modifier categories and selection should occur separately for each, filter to one category before ranking or add that category discriminator to `partition_by`. Incomplete target identities are excluded. The stable source table and modifier ID are always the final tie-breakers under the default contract. + +`modifier_target_queries(..., diagnostics=True)` returns a second advisory query covering `missing_target_identity`, `unsupported_target_field`, `missing_target_event`, and `person_mismatch`. Diagnostics do not change the valid result. For a caller-supplied selectable, only missing identity and observed person mismatches are reported: a filtered result cannot prove that an event is absent from the underlying table or that its Field is unsupported. + +Diagnostics support inspection and validation independently of selection. Consume SQLAlchemy mappings directly, or convert them with the thin typed adapters. The adapters do not execute queries or alter the valid matches: + +```python +from omop_alchemy.toolkit.core.modifiers import ModifierTargetDiagnostic + +checked = modifier_target_queries( + Measurement, Condition_Occurrence, diagnostics=True, +) +assert checked.diagnostics is not None +diagnostic_rows = session.execute(checked.diagnostics).mappings().all() +typed_diagnostics = [ + ModifierTargetDiagnostic.from_mapping(row) for row in diagnostic_rows +] +``` + +`EpisodeAttachmentDiagnostic` provides the corresponding convenience for attachment diagnostics, as shown above. These are exported downstream validation contracts; their presence does not imply an in-package workflow or a scheduled consumer integration. + +Oncology stage preference composes with this generic selector. Its public default is pathological, clinical, then unclassified, followed by earliest time. `StageSelectionSpec.clinical_first()`, `StageSelectionSpec.chronological_only()`, and a `latest` temporal policy are explicit query-scoped alternatives. + +## API reference + +::: omop_alchemy.toolkit.core.events + +::: omop_alchemy.toolkit.core.modifiers + +::: omop_alchemy.toolkit.episodes.derivation diff --git a/mkdocs.yml b/mkdocs.yml index cebc5db..821b051 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -60,6 +60,7 @@ plugins: merge_init_into_class: true separate_signature: true show_signature_annotations: true + heading_level: 4 nav: - Home: index.md @@ -134,10 +135,11 @@ nav: - Toolkit: - Overview: toolkit/index.md - - core: toolkit/core.md - - episodes: toolkit/episodes.md - - analytics: toolkit/analytics.md - - integrations: toolkit/integrations.md + - Core services: toolkit/core.md + - Materialized views: toolkit/materialized-views.md + - Episodes: toolkit/episodes.md + - Query contracts: toolkit/query-contracts.md + - Clinical analytics: toolkit/analytics.md - OMOP-Specific Validation: - Overview: validation/index.md @@ -149,3 +151,4 @@ nav: - Backends: advanced/backends.md - Patient Timelines: advanced/timelines.md - PostgreSQL Full-Text Search: advanced/fulltext.md + - Vocabulary Load Performance: advanced/vocabulary_load_performance.md diff --git a/omop_alchemy/cdm/base/__init__.py b/omop_alchemy/cdm/base/__init__.py index fbbe0b1..0c76798 100644 --- a/omop_alchemy/cdm/base/__init__.py +++ b/omop_alchemy/cdm/base/__init__.py @@ -7,7 +7,8 @@ from .concept_validation import ConceptValidationMixin from .reference_context import ReferenceContext from .typing import HasConceptId, HasEpisodeId, HasPersonId, DomainSemanticTable -from .modifier_interface import ModifierTargetMixin +from .modifier_interface import ModifierSourceMixin, ModifierTargetMixin +from .event_metadata import ClinicalEventMixin from .cdm_constants import ModifierFieldConcepts __all__ = [ @@ -33,7 +34,9 @@ "ConceptValidationMixin", "FactTable", "merge_table_args", + "ModifierSourceMixin", "ModifierTargetMixin", + "ClinicalEventMixin", "ModifierFieldConcepts", "DomainRule", "omop_index", diff --git a/omop_alchemy/cdm/base/cdm_constants.py b/omop_alchemy/cdm/base/cdm_constants.py index 63dea95..139bbe8 100644 --- a/omop_alchemy/cdm/base/cdm_constants.py +++ b/omop_alchemy/cdm/base/cdm_constants.py @@ -1,5 +1,8 @@ class ModifierFieldConcepts: CONDITION_OCCURRENCE = 1147127 + MEASUREMENT = 1147138 + OBSERVATION = 1147165 PROCEDURE_OCCURRENCE = 1147082 DRUG_EXPOSURE = 1147707 + DEVICE_EXPOSURE = 1147693 EPISODE = 756290 diff --git a/omop_alchemy/cdm/base/errors.py b/omop_alchemy/cdm/base/errors.py new file mode 100644 index 0000000..4bdd1be --- /dev/null +++ b/omop_alchemy/cdm/base/errors.py @@ -0,0 +1,17 @@ +"""Shared error contracts for CDM model metadata validation.""" + +from __future__ import annotations + +from typing import ClassVar + + +class UnsupportedModelError(TypeError): + """Base error carrying the model and reason for unsupported-model failures.""" + + model_kind: ClassVar[str] = "model" + + def __init__(self, model: object, reason: str) -> None: + self.model = model + self.reason = reason + name = getattr(model, "__name__", repr(model)) + super().__init__(f"{name} is not a supported {self.model_kind}: {reason}") diff --git a/omop_alchemy/cdm/base/event_metadata.py b/omop_alchemy/cdm/base/event_metadata.py new file mode 100644 index 0000000..eeadcf2 --- /dev/null +++ b/omop_alchemy/cdm/base/event_metadata.py @@ -0,0 +1,119 @@ +"""Database-free event metadata shapes and capability checks.""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any + +from .errors import UnsupportedModelError +from .modifier_interface import ModifierTargetMixin + + +class UnsupportedClinicalEventModelError(UnsupportedModelError): + """Raised when a model cannot provide a canonical clinical-event projection.""" + + model_kind = "clinical-event model" + + +@dataclass(frozen=True, slots=True) +class ClinicalEventModelSpec: + """Resolved model metadata used to build a canonical event projection.""" + + event_id_column: str + event_concept_id_column: str + event_date_column: str + event_datetime_column: str | None + event_field_concept_id: int + event_source_table: str + event_end_date_column: str | None = None + event_end_datetime_column: str | None = None + + +class ClinicalEventMixin(ModifierTargetMixin): + """CDM event metadata and helpers shared by clinical analytical Views. + + Inheritance provides a metadata capability, not registry membership. Custom + modifier sources may use this mixin without becoming episode-event targets. + The model registry selects supported events; toolkit builds SQL from specs. + """ + + @classmethod + def has_complete_metadata(cls) -> bool: + """Whether event identity, concept, date and Field metadata are declared.""" + if any( + not getattr(cls, name, None) + for name in ("__event_id_col__", "__concept_id_col__", "__start_date_col__") + ): + return False + try: + cls.modifier_field_concept_id() + except NotImplementedError: + return False + return True + + @classmethod + def clinical_event_model_spec( + cls, source_model: type[Any] | None = None + ) -> ClinicalEventModelSpec: + """Validate event metadata against this View or a bare source model. + + This method accesses no database or registry. A registered View can + supply declarations while a bare table supplies the physical columns. + """ + model = cls if source_model is None else source_model + if not isinstance(model, type) or not hasattr(model, "__table__"): + raise UnsupportedClinicalEventModelError( + model, "expected a mapped ORM model class" + ) + if not cls.has_complete_metadata(): + raise UnsupportedClinicalEventModelError( + model, "no complete ClinicalEventMixin metadata is available" + ) + + event_id_column = cls.__event_id_col__ + event_concept_id_column = cls.__concept_id_col__ + event_date_column = cls.__start_date_col__ + required_columns = ( + event_id_column, + event_concept_id_column, + event_date_column, + "person_id", + ) + missing = tuple(name for name in required_columns if not hasattr(model, name)) + if missing: + raise UnsupportedClinicalEventModelError( + model, f"missing required columns: {', '.join(missing)}" + ) + + end_date, end_datetime = cls._interval_columns(model) + return ClinicalEventModelSpec( + event_id_column=event_id_column, + event_concept_id_column=event_concept_id_column, + event_date_column=event_date_column, + event_datetime_column=cls._datetime_column_name(model, event_date_column), + event_field_concept_id=cls.modifier_field_concept_id(), + event_source_table=cls.modifier_target_table(), + event_end_date_column=end_date, + event_end_datetime_column=end_datetime, + ) + + @staticmethod + def _datetime_column_name(model: type[Any], date_column_name: str) -> str | None: + """Return the conventional datetime counterpart only when exposed.""" + if date_column_name.endswith("_date"): + candidate = f"{date_column_name[:-5]}_datetime" + if hasattr(model, candidate): + return candidate + return None + + @classmethod + def _interval_columns(cls, model: type[Any]) -> tuple[str | None, str | None]: + """Resolve independent endpoints; a start-date alias denotes a point.""" + end_date = getattr(cls, "__end_date_col__", None) + if ( + not isinstance(end_date, str) + or end_date == cls.__start_date_col__ + or not hasattr(model, end_date) + ): + return None, None + return end_date, cls._datetime_column_name(model, end_date) diff --git a/omop_alchemy/cdm/base/modifier_interface.py b/omop_alchemy/cdm/base/modifier_interface.py index a21f3e0..aee18d5 100644 --- a/omop_alchemy/cdm/base/modifier_interface.py +++ b/omop_alchemy/cdm/base/modifier_interface.py @@ -4,14 +4,80 @@ from datetime import date from sqlalchemy.sql.elements import SQLColumnExpression +class ModifierSourceMixin: + """ + Marker and helpers for OMOP tables that record a supplementary fact about + another CDM row through OMOP's polymorphic modifier link. + + OMOP puts the modifier link on the source table under table-specific + column names (``measurement_event_id`` / ``meas_event_field_concept_id`` + versus ``observation_event_id`` / ``obs_event_field_concept_id``). + Subclasses name those columns once and inherit a common vocabulary, so + query code never branches on the physical modifier source. + + Built-in CDM tables put this mixin on the bare Measurement and Observation + classes. Custom mapped sources combine it with ``ClinicalEventMixin`` to + supply event and link metadata; source support is validated from that metadata, + not restricted to the built-in cached specs. Wearing this mixin does not + enrol a model in the clinical-event or modifier-target registries. + + The mixin exposes row-level link columns and properties only. Bulk target + validation, including person and Field-concept checks, is provided by the + toolkit's ``modifier_target_queries`` builder rather than by an implicit + ``resolved_target`` lookup on each ORM instance. + + This asymmetry is deliberate: Episode_EventView retains a session-bound + convenience for navigating one existing link, without person validation. + Measurement and Observation instead use set-based target queries for bulk + processing and person/Field/event validation, avoiding implicit per-row + target loads on fact-table instances. + """ + + __abstract__ = True + __tablename__: ClassVar[str] + # Source-link metadata names the target identity, not this row's own ID. + __modifier_event_id_col__: ClassVar[str] + __modifier_field_concept_id_col__: ClassVar[str] + + @classmethod + def modifier_source_table(cls) -> str: + return cls.__tablename__ + + @hybrid_property + def modifier_of_event_id(self) -> Optional[int]: + return getattr(self, self.__modifier_event_id_col__) + + @modifier_of_event_id.inplace.expression + @classmethod + def _modifier_of_event_id(cls) -> SQLColumnExpression[Optional[int]]: + return getattr(cls, cls.__modifier_event_id_col__) + + @hybrid_property + def modifier_of_field_concept_id(self) -> Optional[int]: + return getattr(self, self.__modifier_field_concept_id_col__) + + @modifier_of_field_concept_id.inplace.expression + @classmethod + def _modifier_of_field_concept_id(cls) -> SQLColumnExpression[Optional[int]]: + return getattr(cls, cls.__modifier_field_concept_id_col__) + + class ModifierTargetMixin: """ - Marker + helpers for OMOP tables that can be modified - by Measurements / Observations / Episode Events. + Marker and helpers for OMOP tables that can receive a polymorphic modifier + link from Measurements, Observations, or Episode Events. + + Wearing this mixin describes target-row identity metadata; it does not + enrol a class as a clinical event or modifier target in a registry. + + Built-in CDM models place it on analytical Views, keeping the bare tables + lean. That placement convention does not restrict custom mapped sources + from supplying source and event metadata without a View/context base. """ __abstract__ = True __tablename__: ClassVar[str] + # Target-self metadata names this row's identity and clinical fields. __event_id_col__: ClassVar[str] __concept_id_col__: ClassVar[str] __start_date_col__: ClassVar[str] @@ -50,4 +116,4 @@ def end_date(self) -> Optional[date]: @property def type_concept_id(self) -> int: - return getattr(self, self.__type_concept_id_col__) \ No newline at end of file + return getattr(self, self.__type_concept_id_col__) diff --git a/omop_alchemy/cdm/base/reference_context.py b/omop_alchemy/cdm/base/reference_context.py index 6948abe..0d2014e 100644 --- a/omop_alchemy/cdm/base/reference_context.py +++ b/omop_alchemy/cdm/base/reference_context.py @@ -1,34 +1,16 @@ - from __future__ import annotations import sqlalchemy.orm as so import sqlalchemy as sa +from typing import Any -class ReferenceContext: - - """ - `ReferenceContext` - - A helper base class for defining **read-only reference relationships**. - - This class is purely structural: it resolves foreign keys - into reference tables (Domain, Vocabulary, ConceptClass, etc.) - with explicit join conditions. - - These relationships are: - - `viewonly=True` - - explicitly joined - - loaded using `selectin` (batched eager loading) - - defined outside the core table - - deterministic projections of foreign keys - - They are intended for: +class ReferenceContext: + """Read-only reference relationships for inspection and analytics. - - inspection - - analytics - - debugging - - view-level navigation - — **not** for ETL or mutation. + Analytical contexts define these relationships outside the bare ETL table. + Explicit joins resolve local references to single-column target identity, + with viewonly=True and selectin loading for batched navigation. Assigning + a related object does not mutate or populate the local reference column. """ @classmethod @@ -37,19 +19,45 @@ def _reference_relationship( *, target: str, local_fk: str, - remote_pk: str, + remote_pk: str | None = None, uselist: bool = False, ): + """Join a local reference to its target's single mapped primary key. + + Target metadata is resolved only when the join is configured, preserving + declaration/import order. The released ``remote_pk`` argument remains + accepted, but must name the derived primary-key attribute. + """ return so.declared_attr( lambda cls_: so.relationship( target, - primaryjoin=lambda: getattr(cls_, local_fk) == getattr( - sa.inspect(cls_).registry._class_registry[target], - remote_pk, + primaryjoin=lambda: ( + getattr(cls_, local_fk) + == cls._reference_primary_key(cls_, target, remote_pk) ), foreign_keys=lambda: getattr(cls_, local_fk), viewonly=True, lazy="selectin", uselist=uselist, ) - ) \ No newline at end of file + ) + + @staticmethod + def _reference_primary_key( + model: type[Any], target: str, remote_pk: str | None + ) -> so.InstrumentedAttribute[Any]: + target_model = sa.inspect(model).registry._class_registry.get(target) + if not isinstance(target_model, type): + raise ValueError(f"reference target {target!r} is missing or ambiguous") + mapper = sa.inspect(target_model) + if len(mapper.primary_key) != 1: + raise ValueError( + f"reference target {target!r} must have exactly one mapped primary key" + ) + attribute = mapper.get_property_by_column(mapper.primary_key[0]).class_attribute + if remote_pk is not None and remote_pk != attribute.key: + raise ValueError( + f"reference target {target!r} primary key is {attribute.key!r}, " + f"not {remote_pk!r}" + ) + return attribute diff --git a/omop_alchemy/cdm/model/clinical/__init__.py b/omop_alchemy/cdm/model/clinical/__init__.py index 9fe1046..ab59a0f 100644 --- a/omop_alchemy/cdm/model/clinical/__init__.py +++ b/omop_alchemy/cdm/model/clinical/__init__.py @@ -1,24 +1,44 @@ -from .condition_occurrence import Condition_Occurrence, Condition_OccurrenceContext, Condition_OccurrenceView -from .measurement import Measurement -from .observation import Observation +from .condition_occurrence import ( + Condition_Occurrence, + Condition_OccurrenceContext, + Condition_OccurrenceView, +) +from .drug_exposure import Drug_Exposure, Drug_ExposureContext, Drug_ExposureView +from .measurement import Measurement, MeasurementContext, MeasurementView +from .observation import Observation, ObservationContext, ObservationView from .person import Person, PersonView -from .drug_exposure import Drug_Exposure -from .procedure_occurrence import Procedure_Occurrence -from .device_exposure import Device_Exposure +from .procedure_occurrence import ( + Procedure_Occurrence, + Procedure_OccurrenceContext, + Procedure_OccurrenceView, +) +from .device_exposure import ( + Device_Exposure, Device_ExposureContext, Device_ExposureView +) from .death import Death from .specimen import Specimen __all__ = [ - "Condition_Occurrence", - "Condition_OccurrenceContext", + "Condition_Occurrence", + "Condition_OccurrenceContext", "Condition_OccurrenceView", + "Drug_Exposure", + "Drug_ExposureContext", + "Drug_ExposureView", "Measurement", + "MeasurementContext", + "MeasurementView", "Observation", + "ObservationContext", + "ObservationView", "Person", - "Drug_Exposure", "Procedure_Occurrence", + "Procedure_OccurrenceContext", + "Procedure_OccurrenceView", "Device_Exposure", + "Device_ExposureContext", + "Device_ExposureView", "Death", "Specimen", - "PersonView" -] \ No newline at end of file + "PersonView", +] diff --git a/omop_alchemy/cdm/model/clinical/clinical_event_union.py b/omop_alchemy/cdm/model/clinical/clinical_event_union.py index 4c03a15..03018b2 100644 --- a/omop_alchemy/cdm/model/clinical/clinical_event_union.py +++ b/omop_alchemy/cdm/model/clinical/clinical_event_union.py @@ -1,9 +1,20 @@ +import warnings +from typing_extensions import deprecated + from sqlalchemy import select, union_all, literal from .condition_occurrence import Condition_Occurrence -#from .drug_exposure import Drug_Exposure +# from .drug_exposure import Drug_Exposure from orm_loader.helpers import Base +warnings.warn( + "omop_alchemy.cdm.model.clinical.clinical_event_union is deprecated; " + "use omop_alchemy.toolkit.core.events.canonical_event_union instead. " + "The compatibility module will be removed in omop-alchemy 2.0.", + DeprecationWarning, + stacklevel=2, +) + clinical_event_union = union_all( select( literal("condition").label("domain"), @@ -30,6 +41,12 @@ # procedure... ).subquery("clinical_event") + +@deprecated( + "Use omop_alchemy.toolkit.core.events.canonical_event_union instead. " + "ClinicalEventView will be removed in omop-alchemy 2.0.", + category=None, +) class ClinicalEventView(Base): __table__ = clinical_event_union __mapper_args__ = { @@ -37,4 +54,4 @@ class ClinicalEventView(Base): clinical_event_union.c.domain, clinical_event_union.c.event_id, ] - } \ No newline at end of file + } diff --git a/omop_alchemy/cdm/model/clinical/condition_occurrence.py b/omop_alchemy/cdm/model/clinical/condition_occurrence.py index e9160ca..bba3e43 100644 --- a/omop_alchemy/cdm/model/clinical/condition_occurrence.py +++ b/omop_alchemy/cdm/model/clinical/condition_occurrence.py @@ -12,7 +12,7 @@ CDMTableBase, cdm_table, ModifierFieldConcepts, - ModifierTargetMixin, + ClinicalEventMixin, merge_table_args, omop_index, ) @@ -49,10 +49,10 @@ class Condition_Occurrence( condition_status_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) class Condition_OccurrenceContext(ReferenceContext): - condition_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="condition_concept_id", remote_pk="concept_id") # type: ignore[assignment] - condition_type: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="condition_type_concept_id", remote_pk="concept_id") # type: ignore[assignment] - condition_source_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="condition_source_concept_id", remote_pk="concept_id") # type: ignore[assignment] - condition_status: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="condition_status_concept_id", remote_pk="concept_id") # type: ignore[assignment] + condition_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="condition_concept_id") # type: ignore[assignment] + condition_type: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="condition_type_concept_id") # type: ignore[assignment] + condition_source_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="condition_source_concept_id") # type: ignore[assignment] + condition_status: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="condition_status_concept_id") # type: ignore[assignment] @declared_attr def visit_occurrence(cls) -> so.Mapped[Optional["Visit_Occurrence"]]: @@ -67,7 +67,7 @@ def visit_occurrence(cls) -> so.Mapped[Optional["Visit_Occurrence"]]: class Condition_OccurrenceView( Condition_Occurrence, Condition_OccurrenceContext, - ModifierTargetMixin + ClinicalEventMixin ): __tablename__ = "condition_occurrence" __mapper_args__ = {"concrete": False} diff --git a/omop_alchemy/cdm/model/clinical/death.py b/omop_alchemy/cdm/model/clinical/death.py index 85daeed..602af29 100644 --- a/omop_alchemy/cdm/model/clinical/death.py +++ b/omop_alchemy/cdm/model/clinical/death.py @@ -35,9 +35,9 @@ class Death(CDMTableBase, Base): class DeathContext(ReferenceContext): - person: so.Mapped["Person"] = ReferenceContext._reference_relationship(target="Person",local_fk="person_id",remote_pk="person_id") # type: ignore[assignment] - death_type_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="death_type_concept_id", remote_pk="concept_id") # type: ignore[assignment] - cause_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="cause_concept_id", remote_pk="concept_id") # type: ignore[assignment] + person: so.Mapped["Person"] = ReferenceContext._reference_relationship(target="Person",local_fk="person_id") # type: ignore[assignment] + death_type_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="death_type_concept_id") # type: ignore[assignment] + cause_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="cause_concept_id") # type: ignore[assignment] class DeathView(Death, DeathContext, DomainValidationMixin): __tablename__ = "death" diff --git a/omop_alchemy/cdm/model/clinical/device_exposure.py b/omop_alchemy/cdm/model/clinical/device_exposure.py index c4862b2..4a77e7d 100644 --- a/omop_alchemy/cdm/model/clinical/device_exposure.py +++ b/omop_alchemy/cdm/model/clinical/device_exposure.py @@ -1,6 +1,6 @@ import sqlalchemy as sa import sqlalchemy.orm as so -from typing import Optional +from typing import Optional, TYPE_CHECKING from datetime import date, datetime from orm_loader.helpers import Base @@ -9,21 +9,29 @@ HealthSystemContext, FactTable, CDMTableBase, + DomainValidationMixin, + ExpectedDomain, + ModifierFieldConcepts, + ReferenceContext, cdm_table, required_concept_fk, optional_concept_fk, optional_int, - ModifierTargetMixin, + ClinicalEventMixin, merge_table_args, omop_index, ) +if TYPE_CHECKING: + from ..health_system import Provider, Visit_Detail, Visit_Occurrence + from ..vocabulary import Concept + from .person import Person + @cdm_table class Device_Exposure( PersonScoped, CDMTableBase, FactTable, - ModifierTargetMixin, HealthSystemContext, Base, ): @@ -52,3 +60,79 @@ class Device_Exposure( quantity: so.Mapped[Optional[int]] = optional_int() device_source_value: so.Mapped[Optional[str]] = so.mapped_column(sa.String(50)) unit_source_value: so.Mapped[Optional[str]] = so.mapped_column(sa.String(50)) + + +class Device_ExposureContext(ReferenceContext): + """Read-only analytical relationships for a Device Exposure row.""" + + person: so.Mapped["Person"] = ReferenceContext._reference_relationship( + target="Person", local_fk="person_id" + ) # type: ignore[assignment] + device_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship( + target="Concept", local_fk="device_concept_id" + ) # type: ignore[assignment] + device_type_concept: so.Mapped["Concept"] = ( + ReferenceContext._reference_relationship( + target="Concept", + local_fk="device_type_concept_id", + ) + ) # type: ignore[assignment] + device_source_concept: so.Mapped[Optional["Concept"]] = ( + ReferenceContext._reference_relationship( + target="Concept", + local_fk="device_source_concept_id", + ) + ) # type: ignore[assignment] + unit_concept: so.Mapped[Optional["Concept"]] = ( + ReferenceContext._reference_relationship( + target="Concept", local_fk="unit_concept_id" + ) + ) # type: ignore[assignment] + unit_source_concept: so.Mapped[Optional["Concept"]] = ( + ReferenceContext._reference_relationship( + target="Concept", + local_fk="unit_source_concept_id", + ) + ) # type: ignore[assignment] + provider: so.Mapped[Optional["Provider"]] = ( + ReferenceContext._reference_relationship( + target="Provider", local_fk="provider_id" + ) + ) # type: ignore[assignment] + visit_occurrence: so.Mapped[Optional["Visit_Occurrence"]] = ( + ReferenceContext._reference_relationship( + target="Visit_Occurrence", + local_fk="visit_occurrence_id", + ) + ) # type: ignore[assignment] + visit_detail: so.Mapped[Optional["Visit_Detail"]] = ( + ReferenceContext._reference_relationship( + target="Visit_Detail", + local_fk="visit_detail_id", + ) + ) # type: ignore[assignment] + + +class Device_ExposureView( + Device_Exposure, + Device_ExposureContext, + DomainValidationMixin, + ClinicalEventMixin, +): + """Analytical Device Exposure mapping with event metadata and references.""" + + __tablename__ = "device_exposure" + __mapper_args__ = {"concrete": False} + __event_id_col__ = "device_exposure_id" + __concept_id_col__ = "device_concept_id" + __start_date_col__ = "device_exposure_start_date" + __end_date_col__ = "device_exposure_end_date" + __type_concept_id_col__ = "device_type_concept_id" + __expected_domains__ = { + "device_concept_id": ExpectedDomain("Device"), + "device_type_concept_id": ExpectedDomain("Type Concept"), + } + + @classmethod + def modifier_field_concept_id(cls) -> int: + return ModifierFieldConcepts.DEVICE_EXPOSURE diff --git a/omop_alchemy/cdm/model/clinical/drug_exposure.py b/omop_alchemy/cdm/model/clinical/drug_exposure.py index ea2d9aa..621401d 100644 --- a/omop_alchemy/cdm/model/clinical/drug_exposure.py +++ b/omop_alchemy/cdm/model/clinical/drug_exposure.py @@ -13,7 +13,7 @@ required_concept_fk, optional_concept_fk, optional_int, - ModifierTargetMixin, + ClinicalEventMixin, ModifierFieldConcepts, merge_table_args, omop_index, @@ -27,7 +27,6 @@ class Drug_Exposure( PersonScoped, CDMTableBase, FactTable, - ModifierTargetMixin, HealthSystemContext, Base, ): @@ -64,16 +63,16 @@ class Drug_Exposure( class Drug_ExposureContext(ReferenceContext): - drug_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="drug_concept_id", remote_pk="concept_id") # type: ignore[assignment] - drug_type: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="drug_type_concept_id", remote_pk="concept_id") # type: ignore[assignment] - route: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="route_concept_id", remote_pk="concept_id") # type: ignore[assignment] - drug_source_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="drug_source_concept_id", remote_pk="concept_id") # type: ignore[assignment] + drug_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="drug_concept_id") # type: ignore[assignment] + drug_type: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="drug_type_concept_id") # type: ignore[assignment] + route: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="route_concept_id") # type: ignore[assignment] + drug_source_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept", local_fk="drug_source_concept_id") # type: ignore[assignment] class Drug_ExposureView( Drug_Exposure, Drug_ExposureContext, - ModifierTargetMixin + ClinicalEventMixin ): __tablename__ = "drug_exposure" diff --git a/omop_alchemy/cdm/model/clinical/event_metadata.py b/omop_alchemy/cdm/model/clinical/event_metadata.py new file mode 100644 index 0000000..c1d2cf6 --- /dev/null +++ b/omop_alchemy/cdm/model/clinical/event_metadata.py @@ -0,0 +1,160 @@ +"""Stable metadata for CDM tables that participate in clinical-event APIs. + +Being a clinical event and being a valid modifier target are different +questions with different membership. Every clinical event is a valid modifier +target, but not every modifier target is a clinical event. +""" + +from __future__ import annotations + +from collections.abc import Callable +from types import MappingProxyType +from typing import Any, Mapping + +from omop_alchemy.cdm.base import ( + ClinicalEventMixin, + ModifierSourceMixin, + ModifierTargetMixin, +) +from omop_alchemy.cdm.base.event_metadata import ( + ClinicalEventModelSpec, + UnsupportedClinicalEventModelError, +) + +from .condition_occurrence import Condition_Occurrence, Condition_OccurrenceView +from .device_exposure import Device_Exposure, Device_ExposureView +from .drug_exposure import Drug_Exposure, Drug_ExposureView +from .measurement import Measurement, MeasurementView +from .observation import Observation, ObservationView +from .procedure_occurrence import Procedure_Occurrence, Procedure_OccurrenceView +from ..structural.episode import Episode, EpisodeView + + +# Keep one explicit supported event set. Both lookup shapes are derived from it +# so projection and episode-resolution support cannot drift independently. +_CLINICAL_EVENT_TARGETS: tuple[tuple[type[Any], type[ClinicalEventMixin]], ...] = ( + (Condition_Occurrence, Condition_OccurrenceView), + (Device_Exposure, Device_ExposureView), + (Drug_Exposure, Drug_ExposureView), + (Measurement, MeasurementView), + (Observation, ObservationView), + (Procedure_Occurrence, Procedure_OccurrenceView), + # Episode is a valid modifier target, but it is not a clinical event and is + # therefore deliberately absent from this registry. +) + +_STRUCTURAL_MODIFIER_TARGETS: tuple[ + tuple[type[Any], type[ModifierTargetMixin]], ... +] = ((Episode, EpisodeView),) + + +def _validate_unique_target_keys( + entries: tuple[tuple[type[Any], type[ModifierTargetMixin]], ...], + *, + key: Callable[[type[Any], type[ModifierTargetMixin]], object], + label: str, +) -> None: + """Reject duplicate target identities before a registry becomes immutable.""" + seen: dict[object, str] = {} + for source, target in entries: + identity = key(source, target) + previous = seen.get(identity) + if previous is not None: + raise ValueError( + f"duplicate {label} {identity!r}: {previous} and {target.__name__}" + ) + seen[identity] = target.__name__ + + +_ALL_MODIFIER_TARGETS = _CLINICAL_EVENT_TARGETS + _STRUCTURAL_MODIFIER_TARGETS +_validate_unique_target_keys( + _ALL_MODIFIER_TARGETS, + key=lambda source, _target: source.__tablename__, + label="modifier target table", +) +_validate_unique_target_keys( + _ALL_MODIFIER_TARGETS, + key=lambda _source, target: target.modifier_field_concept_id(), + label="modifier field concept ID", +) + +CLINICAL_EVENT_TARGETS_BY_TABLE: Mapping[str, type[ClinicalEventMixin]] = ( + MappingProxyType( + {source.__tablename__: target for source, target in _CLINICAL_EVENT_TARGETS} + ) +) +CLINICAL_EVENT_TARGETS_BY_FIELD_CONCEPT_ID: Mapping[int, type[ClinicalEventMixin]] = ( + MappingProxyType( + { + target.modifier_field_concept_id(): target + for _, target in _CLINICAL_EVENT_TARGETS + } + ) +) + +STRUCTURAL_MODIFIER_TARGETS_BY_TABLE: Mapping[str, type[ModifierTargetMixin]] = ( + MappingProxyType( + { + source.__tablename__: target + for source, target in _STRUCTURAL_MODIFIER_TARGETS + } + ) +) + +MODIFIER_TARGETS_BY_TABLE: Mapping[str, type[ModifierTargetMixin]] = MappingProxyType( + {**CLINICAL_EVENT_TARGETS_BY_TABLE, **STRUCTURAL_MODIFIER_TARGETS_BY_TABLE} +) + + +def clinical_event_target_for_table( + table_name: str, +) -> type[ClinicalEventMixin] | None: + """Return the registered analytical event target for a bare CDM table.""" + return CLINICAL_EVENT_TARGETS_BY_TABLE.get(table_name) + + +def _metadata_candidate(model: type[Any]) -> type[ClinicalEventMixin] | None: + # An explicitly supplied registered event view (or its domain-specific + # subclass) owns its metadata. Bare CDM tables use the registered CDM view + # for that table; unrelated subclasses are never discovered by walking + # Python's import-dependent subclass graph. + if issubclass(model, ModifierTargetMixin): + if ( + not issubclass(model, ClinicalEventMixin) + or not model.has_complete_metadata() + ): + return None + table_name = getattr(model, "__tablename__", None) + registered_view = clinical_event_target_for_table(str(table_name)) + if registered_view is not None and issubclass(model, registered_view): + return model + # Modifier sources outside the built-in event set are intentionally + # supported by the canonical modifier projection. They carry the same + # event-shaped metadata, but do not become episode-resolvable events. + if issubclass(model, ModifierSourceMixin): + return model + return None + # Lean CDM models intentionally do not carry modifier metadata. Resolve + # their table through the configured analytical view without changing the + # class used to read scalar event rows. + table_name = getattr(model, "__tablename__", None) + return clinical_event_target_for_table(str(table_name)) + + +def clinical_event_model_spec(model: type[Any]) -> ClinicalEventModelSpec: + """Resolve the event metadata for an ORM model without accessing a database.""" + # Resolve metadata before building SQL so unsupported models fail at query + # construction, rather than producing a partially shaped union at runtime. + if not isinstance(model, type) or not hasattr(model, "__table__"): + raise UnsupportedClinicalEventModelError( + model, "expected a mapped ORM model class" + ) + + metadata_model = _metadata_candidate(model) + if metadata_model is None: + raise UnsupportedClinicalEventModelError( + model, + "no complete ClinicalEventMixin metadata is available", + ) + + return metadata_model.clinical_event_model_spec(model) diff --git a/omop_alchemy/cdm/model/clinical/measurement.py b/omop_alchemy/cdm/model/clinical/measurement.py index aa94954..3342d86 100644 --- a/omop_alchemy/cdm/model/clinical/measurement.py +++ b/omop_alchemy/cdm/model/clinical/measurement.py @@ -2,20 +2,31 @@ import sqlalchemy as sa import sqlalchemy.orm as so -from sqlalchemy.ext.hybrid import hybrid_property -from typing import Optional +from typing import Optional, TYPE_CHECKING from datetime import date, datetime from orm_loader.helpers import Base from omop_alchemy.cdm.base import ( CDMTableBase, + DomainValidationMixin, + ExpectedDomain, + ModifierFieldConcepts, + ModifierSourceMixin, + ClinicalEventMixin, + ReferenceContext, cdm_table, ValueMixin, merge_table_args, omop_index, ) +if TYPE_CHECKING: + from ..health_system import Provider, Visit_Detail, Visit_Occurrence + from ..vocabulary import Concept + from .person import Person + + @cdm_table -class Measurement(Base, CDMTableBase, ValueMixin): +class Measurement(Base, CDMTableBase, ValueMixin, ModifierSourceMixin): __tablename__ = "measurement" __table_args__ = merge_table_args( omop_index(__tablename__, "person_id", cluster=True), @@ -26,35 +37,127 @@ class Measurement(Base, CDMTableBase, ValueMixin): ) measurement_id: so.Mapped[int] = so.mapped_column(primary_key=True) - person_id: so.Mapped[int] = so.mapped_column(sa.ForeignKey("person.person_id"), nullable=False) - measurement_concept_id: so.Mapped[int] = so.mapped_column(sa.ForeignKey("concept.concept_id"), nullable=False) + person_id: so.Mapped[int] = so.mapped_column( + sa.ForeignKey("person.person_id"), nullable=False + ) + measurement_concept_id: so.Mapped[int] = so.mapped_column( + sa.ForeignKey("concept.concept_id"), nullable=False + ) measurement_date: so.Mapped[date] = so.mapped_column(nullable=False) measurement_datetime: so.Mapped[Optional[datetime]] measurement_time: so.Mapped[Optional[str]] - measurement_type_concept_id: so.Mapped[int] = so.mapped_column(sa.ForeignKey("concept.concept_id"), nullable=False) - operator_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) - unit_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) + measurement_type_concept_id: so.Mapped[int] = so.mapped_column( + sa.ForeignKey("concept.concept_id"), nullable=False + ) + operator_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id") + ) + unit_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id") + ) range_low: so.Mapped[Optional[float]] range_high: so.Mapped[Optional[float]] - provider_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("provider.provider_id")) - visit_occurrence_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("visit_occurrence.visit_occurrence_id")) - visit_detail_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("visit_detail.visit_detail_id")) + provider_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("provider.provider_id") + ) + visit_occurrence_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("visit_occurrence.visit_occurrence_id") + ) + visit_detail_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("visit_detail.visit_detail_id") + ) measurement_source_value: so.Mapped[Optional[str]] - measurement_source_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) + measurement_source_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id") + ) unit_source_value: so.Mapped[Optional[str]] - unit_source_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) + unit_source_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id") + ) value_source_value: so.Mapped[Optional[str]] measurement_event_id: so.Mapped[Optional[int]] - meas_event_field_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id"), doc="Identifies which OMOP table measurement_event_id refers to",) + meas_event_field_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id"), + doc="Identifies which OMOP table measurement_event_id refers to", + ) + + __modifier_event_id_col__ = "measurement_event_id" + __modifier_field_concept_id_col__ = "meas_event_field_concept_id" + - @hybrid_property - def modifier_of_event_id(self) -> Optional[int]: - return self.measurement_event_id +class MeasurementContext(ReferenceContext): + """Read-only analytical relationships for a Measurement row.""" + + person: so.Mapped["Person"] = ReferenceContext._reference_relationship( + target="Person", local_fk="person_id" + ) # type: ignore[assignment] + measurement_concept: so.Mapped["Concept"] = ( + ReferenceContext._reference_relationship( + target="Concept", local_fk="measurement_concept_id" + ) + ) # type: ignore[assignment] + measurement_type_concept: so.Mapped["Concept"] = ( + ReferenceContext._reference_relationship( + target="Concept", + local_fk="measurement_type_concept_id", + ) + ) # type: ignore[assignment] + unit_concept: so.Mapped[Optional["Concept"]] = ( + ReferenceContext._reference_relationship( + target="Concept", local_fk="unit_concept_id" + ) + ) # type: ignore[assignment] + unit_source_concept: so.Mapped[Optional["Concept"]] = ( + ReferenceContext._reference_relationship( + target="Concept", + local_fk="unit_source_concept_id", + ) + ) # type: ignore[assignment] + provider: so.Mapped[Optional["Provider"]] = ( + ReferenceContext._reference_relationship( + target="Provider", local_fk="provider_id" + ) + ) # type: ignore[assignment] + visit_occurrence: so.Mapped[Optional["Visit_Occurrence"]] = ( + ReferenceContext._reference_relationship( + target="Visit_Occurrence", + local_fk="visit_occurrence_id", + ) + ) # type: ignore[assignment] + visit_detail: so.Mapped[Optional["Visit_Detail"]] = ( + ReferenceContext._reference_relationship( + target="Visit_Detail", + local_fk="visit_detail_id", + ) + ) # type: ignore[assignment] + + +class MeasurementView( + Measurement, + MeasurementContext, + DomainValidationMixin, + ClinicalEventMixin, +): + """Analytical Measurement mapping with event metadata and reference context.""" + + __tablename__ = "measurement" + __mapper_args__ = {"concrete": False} + __event_id_col__ = "measurement_id" + __concept_id_col__ = "measurement_concept_id" + __start_date_col__ = "measurement_date" + # One date column: the target API aliases it; interval metadata treats + # equal start/end declarations as a point with no independent endpoint. + __end_date_col__ = "measurement_date" + __type_concept_id_col__ = "measurement_type_concept_id" + __expected_domains__ = { + "measurement_concept_id": ExpectedDomain("Measurement"), + "measurement_type_concept_id": ExpectedDomain("Type Concept"), + } - @hybrid_property - def modifier_of_field_concept_id(self) -> Optional[int]: - return self.meas_event_field_concept_id + @classmethod + def modifier_field_concept_id(cls) -> int: + return ModifierFieldConcepts.MEASUREMENT diff --git a/omop_alchemy/cdm/model/clinical/observation.py b/omop_alchemy/cdm/model/clinical/observation.py index e71bab4..abd1ee8 100644 --- a/omop_alchemy/cdm/model/clinical/observation.py +++ b/omop_alchemy/cdm/model/clinical/observation.py @@ -2,20 +2,31 @@ import sqlalchemy as sa import sqlalchemy.orm as so -from sqlalchemy.ext.hybrid import hybrid_property -from typing import Optional +from typing import Optional, TYPE_CHECKING from datetime import date, datetime from orm_loader.helpers import Base from omop_alchemy.cdm.base import ( CDMTableBase, + DomainValidationMixin, + ExpectedDomain, + ModifierFieldConcepts, + ModifierSourceMixin, + ClinicalEventMixin, + ReferenceContext, cdm_table, ValueMixin, merge_table_args, omop_index, ) +if TYPE_CHECKING: + from ..health_system import Provider, Visit_Detail, Visit_Occurrence + from ..vocabulary import Concept + from .person import Person + + @cdm_table -class Observation(Base, CDMTableBase, ValueMixin): +class Observation(Base, CDMTableBase, ValueMixin, ModifierSourceMixin): __tablename__ = "observation" __table_args__ = merge_table_args( omop_index(__tablename__, "person_id", cluster=True), @@ -25,31 +36,113 @@ class Observation(Base, CDMTableBase, ValueMixin): ) observation_id: so.Mapped[int] = so.mapped_column(primary_key=True) - person_id: so.Mapped[int] = so.mapped_column(sa.ForeignKey("person.person_id"), nullable=False) - observation_concept_id: so.Mapped[int] = so.mapped_column(sa.ForeignKey("concept.concept_id"), nullable=False) + person_id: so.Mapped[int] = so.mapped_column( + sa.ForeignKey("person.person_id"), nullable=False + ) + observation_concept_id: so.Mapped[int] = so.mapped_column( + sa.ForeignKey("concept.concept_id"), nullable=False + ) observation_date: so.Mapped[date] = so.mapped_column(nullable=False) observation_datetime: so.Mapped[Optional[datetime]] - observation_type_concept_id: so.Mapped[int] = so.mapped_column(sa.ForeignKey("concept.concept_id"), nullable=False) - #value_as_number: so.Mapped[Optional[float]] + observation_type_concept_id: so.Mapped[int] = so.mapped_column( + sa.ForeignKey("concept.concept_id"), nullable=False + ) + # value_as_number: so.Mapped[Optional[float]] value_as_string: so.Mapped[Optional[str]] - #value_as_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) - qualifier_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) - unit_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) - provider_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("provider.provider_id")) - visit_occurrence_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("visit_occurrence.visit_occurrence_id")) - visit_detail_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("visit_detail.visit_detail_id")) + # value_as_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) + qualifier_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id") + ) + unit_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id") + ) + provider_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("provider.provider_id") + ) + visit_occurrence_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("visit_occurrence.visit_occurrence_id") + ) + visit_detail_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("visit_detail.visit_detail_id") + ) observation_source_value: so.Mapped[Optional[str]] - observation_source_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) + observation_source_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id") + ) unit_source_value: so.Mapped[Optional[str]] qualifier_source_value: so.Mapped[Optional[str]] value_source_value: so.Mapped[Optional[str]] observation_event_id: so.Mapped[Optional[int]] - obs_event_field_concept_id: so.Mapped[Optional[int]] = so.mapped_column(sa.ForeignKey("concept.concept_id")) + obs_event_field_concept_id: so.Mapped[Optional[int]] = so.mapped_column( + sa.ForeignKey("concept.concept_id") + ) + + __modifier_event_id_col__ = "observation_event_id" + __modifier_field_concept_id_col__ = "obs_event_field_concept_id" + + +class ObservationContext(ReferenceContext): + """Read-only analytical relationships for an Observation row.""" - @hybrid_property - def modifier_of_event_id(self) -> Optional[int]: - return self.observation_event_id + person: so.Mapped["Person"] = ReferenceContext._reference_relationship( + target="Person", local_fk="person_id" + ) # type: ignore[assignment] + observation_concept: so.Mapped["Concept"] = ( + ReferenceContext._reference_relationship( + target="Concept", local_fk="observation_concept_id" + ) + ) # type: ignore[assignment] + observation_type_concept: so.Mapped["Concept"] = ( + ReferenceContext._reference_relationship( + target="Concept", + local_fk="observation_type_concept_id", + ) + ) # type: ignore[assignment] + unit_concept: so.Mapped[Optional["Concept"]] = ( + ReferenceContext._reference_relationship( + target="Concept", local_fk="unit_concept_id" + ) + ) # type: ignore[assignment] + provider: so.Mapped[Optional["Provider"]] = ( + ReferenceContext._reference_relationship( + target="Provider", local_fk="provider_id" + ) + ) # type: ignore[assignment] + visit_occurrence: so.Mapped[Optional["Visit_Occurrence"]] = ( + ReferenceContext._reference_relationship( + target="Visit_Occurrence", + local_fk="visit_occurrence_id", + ) + ) # type: ignore[assignment] + visit_detail: so.Mapped[Optional["Visit_Detail"]] = ( + ReferenceContext._reference_relationship( + target="Visit_Detail", + local_fk="visit_detail_id", + ) + ) # type: ignore[assignment] + + +class ObservationView( + Observation, + ObservationContext, + DomainValidationMixin, + ClinicalEventMixin, +): + """Analytical Observation mapping with event metadata and reference context.""" + + __tablename__ = "observation" + __mapper_args__ = {"concrete": False} + __event_id_col__ = "observation_id" + __concept_id_col__ = "observation_concept_id" + __start_date_col__ = "observation_date" + # One date column: the target API aliases it; interval metadata treats + # equal start/end declarations as a point with no independent endpoint. + __end_date_col__ = "observation_date" + __type_concept_id_col__ = "observation_type_concept_id" + __expected_domains__ = { + "observation_type_concept_id": ExpectedDomain("Type Concept"), + } - @hybrid_property - def modifier_of_field_concept_id(self) -> Optional[int]: - return self.obs_event_field_concept_id + @classmethod + def modifier_field_concept_id(cls) -> int: + return ModifierFieldConcepts.OBSERVATION diff --git a/omop_alchemy/cdm/model/clinical/person.py b/omop_alchemy/cdm/model/clinical/person.py index a22b05b..d5c5153 100644 --- a/omop_alchemy/cdm/model/clinical/person.py +++ b/omop_alchemy/cdm/model/clinical/person.py @@ -65,12 +65,12 @@ def __repr__(self) -> str: return f"" class PersonContext(ReferenceContext): - gender: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="gender_concept_id",remote_pk="concept_id") # type: ignore[assignment] - race: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="race_concept_id",remote_pk="concept_id") # type: ignore[assignment] - ethnicity: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="ethnicity_concept_id",remote_pk="concept_id") # type: ignore[assignment] - location: so.Mapped["Location"] = ReferenceContext._reference_relationship(target="Location",local_fk="location_id",remote_pk="location_id") # type: ignore[assignment] - provider: so.Mapped["Provider"] = ReferenceContext._reference_relationship(target="Provider",local_fk="provider_id",remote_pk="provider_id") # type: ignore[assignment] - care_site: so.Mapped["Care_Site"] = ReferenceContext._reference_relationship(target="Care_Site",local_fk="care_site_id",remote_pk="care_site_id") # type: ignore[assignment] + gender: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="gender_concept_id") # type: ignore[assignment] + race: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="race_concept_id") # type: ignore[assignment] + ethnicity: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="ethnicity_concept_id") # type: ignore[assignment] + location: so.Mapped["Location"] = ReferenceContext._reference_relationship(target="Location",local_fk="location_id") # type: ignore[assignment] + provider: so.Mapped["Provider"] = ReferenceContext._reference_relationship(target="Provider",local_fk="provider_id") # type: ignore[assignment] + care_site: so.Mapped["Care_Site"] = ReferenceContext._reference_relationship(target="Care_Site",local_fk="care_site_id") # type: ignore[assignment] @declared_attr def death(cls) -> so.Mapped[Optional["Death"]]: diff --git a/omop_alchemy/cdm/model/clinical/procedure_occurrence.py b/omop_alchemy/cdm/model/clinical/procedure_occurrence.py index 4dfb65b..4b4cbfa 100644 --- a/omop_alchemy/cdm/model/clinical/procedure_occurrence.py +++ b/omop_alchemy/cdm/model/clinical/procedure_occurrence.py @@ -15,7 +15,7 @@ ReferenceContext, DomainValidationMixin, ExpectedDomain, - ModifierTargetMixin, + ClinicalEventMixin, ModifierFieldConcepts, merge_table_args, omop_index, @@ -55,40 +55,34 @@ class Procedure_OccurrenceContext(ReferenceContext): person: so.Mapped["Person"] = ReferenceContext._reference_relationship( target="Person", local_fk="person_id", - remote_pk="person_id", ) # type: ignore[assignment] procedure_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship( target="Concept", local_fk="procedure_concept_id", - remote_pk="concept_id", ) # type: ignore[assignment] procedure_type_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship( target="Concept", local_fk="procedure_type_concept_id", - remote_pk="concept_id", ) # type: ignore[assignment] modifier_concept: so.Mapped[Optional["Concept"]] = ( ReferenceContext._reference_relationship( target="Concept", local_fk="modifier_concept_id", - remote_pk="concept_id", ) ) # type: ignore[assignment] provider: so.Mapped[Optional["Provider"]] = ReferenceContext._reference_relationship( target="Provider", local_fk="provider_id", - remote_pk="provider_id", ) # type: ignore[assignment] visit_occurrence: so.Mapped[Optional["Visit_Occurrence"]] = ( ReferenceContext._reference_relationship( target="Visit_Occurrence", local_fk="visit_occurrence_id", - remote_pk="visit_occurrence_id", ) ) # type: ignore[assignment] @@ -96,7 +90,6 @@ class Procedure_OccurrenceContext(ReferenceContext): ReferenceContext._reference_relationship( target="Visit_Detail", local_fk="visit_detail_id", - remote_pk="visit_detail_id", ) ) # type: ignore[assignment] @@ -104,7 +97,6 @@ class Procedure_OccurrenceContext(ReferenceContext): ReferenceContext._reference_relationship( target="Concept", local_fk="procedure_source_concept_id", - remote_pk="concept_id", ) ) # type: ignore[assignment] @@ -113,7 +105,7 @@ class Procedure_OccurrenceView( Procedure_Occurrence, Procedure_OccurrenceContext, DomainValidationMixin, - ModifierTargetMixin, + ClinicalEventMixin, ): __tablename__ = "procedure_occurrence" __mapper_args__ = {"concrete": False} diff --git a/omop_alchemy/cdm/model/health_system/visit_occurrence.py b/omop_alchemy/cdm/model/health_system/visit_occurrence.py index 4167219..409b022 100644 --- a/omop_alchemy/cdm/model/health_system/visit_occurrence.py +++ b/omop_alchemy/cdm/model/health_system/visit_occurrence.py @@ -62,9 +62,9 @@ def __repr__(self) -> str: class VisitContext(ReferenceContext): - person: so.Mapped["Person"] = ReferenceContext._reference_relationship(target="Person", local_fk="person_id", remote_pk="person_id",) # type: ignore[assignment] - provider: so.Mapped["Provider"] = ReferenceContext._reference_relationship(target="Provider",local_fk="provider_id",remote_pk="provider_id",) # type: ignore[assignment] - care_site: so.Mapped["Care_Site"] = ReferenceContext._reference_relationship(target="Care_Site",local_fk="care_site_id",remote_pk="care_site_id",) # type: ignore[assignment] + person: so.Mapped["Person"] = ReferenceContext._reference_relationship(target="Person", local_fk="person_id",) # type: ignore[assignment] + provider: so.Mapped["Provider"] = ReferenceContext._reference_relationship(target="Provider",local_fk="provider_id",) # type: ignore[assignment] + care_site: so.Mapped["Care_Site"] = ReferenceContext._reference_relationship(target="Care_Site",local_fk="care_site_id",) # type: ignore[assignment] @declared_attr def procedure_providers(cls) -> so.Mapped[list["Provider"]]: diff --git a/omop_alchemy/cdm/model/structural/episode.py b/omop_alchemy/cdm/model/structural/episode.py index fcab297..eba6cfe 100644 --- a/omop_alchemy/cdm/model/structural/episode.py +++ b/omop_alchemy/cdm/model/structural/episode.py @@ -16,6 +16,8 @@ ExpectedDomain, merge_table_args, omop_index, + ModifierTargetMixin, + ModifierFieldConcepts, ) if TYPE_CHECKING: @@ -58,11 +60,11 @@ def __repr__(self) -> str: class EpisodeContext(ReferenceContext): __table__: ClassVar[sa.Table] - person: so.Mapped["Person"] = ReferenceContext._reference_relationship(target="Person",local_fk="person_id",remote_pk="person_id") # type: ignore[assignment] - episode_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="episode_concept_id",remote_pk="concept_id") # type: ignore[assignment] - episode_object_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="episode_object_concept_id",remote_pk="concept_id") # type: ignore[assignment] - episode_type_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="episode_type_concept_id",remote_pk="concept_id") # type: ignore[assignment] - #parent_episode: so.Mapped[Optional["Episode"]] = ReferenceContext._reference_relationship(target="Episode",local_fk="episode_parent_id",remote_pk="episode_id") # type: ignore[assignment] + person: so.Mapped["Person"] = ReferenceContext._reference_relationship(target="Person",local_fk="person_id") # type: ignore[assignment] + episode_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="episode_concept_id") # type: ignore[assignment] + episode_object_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="episode_object_concept_id") # type: ignore[assignment] + episode_type_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="episode_type_concept_id") # type: ignore[assignment] + #parent_episode: so.Mapped[Optional["Episode"]] = ReferenceContext._reference_relationship(target="Episode",local_fk="episode_parent_id") # type: ignore[assignment] @declared_attr def episode_events(cls: type['HasEpisodeId']) -> so.Mapped[List["Episode_EventView"]]: @@ -99,7 +101,7 @@ def children(cls) -> so.Mapped[List["Episode"]]: uselist=True, ) -class EpisodeView(Episode, EpisodeContext, DomainValidationMixin): +class EpisodeView(Episode, EpisodeContext, DomainValidationMixin, ModifierTargetMixin): """ Navigable Episode view. @@ -111,6 +113,15 @@ class EpisodeView(Episode, EpisodeContext, DomainValidationMixin): __tablename__ = "episode" __mapper_args__ = {"concrete": False} + __event_id_col__ = "episode_id" + __concept_id_col__ = "episode_concept_id" + __start_date_col__ = "episode_start_date" + __end_date_col__ = "episode_end_date" + __type_concept_id_col__ = "episode_type_concept_id" + + @classmethod + def modifier_field_concept_id(cls) -> int: + return ModifierFieldConcepts.EPISODE __expected_domains__ = { "episode_concept_id": ExpectedDomain("Episode"), diff --git a/omop_alchemy/cdm/model/structural/episode_event.py b/omop_alchemy/cdm/model/structural/episode_event.py index 3720c4d..f700aa7 100644 --- a/omop_alchemy/cdm/model/structural/episode_event.py +++ b/omop_alchemy/cdm/model/structural/episode_event.py @@ -1,77 +1,28 @@ import sqlalchemy as sa import sqlalchemy.orm as so -from sqlalchemy import event -from sqlalchemy.orm import Mapper from typing import TYPE_CHECKING, Any, Type -from functools import cached_property, cache +from functools import cached_property from orm_loader.helpers import Base from omop_alchemy.cdm.base import ( cdm_table, CDMTableBase, - MODEL_MODULE_PREFIX, ReferenceContext, DomainValidationMixin, ExpectedDomain, - ModifierTargetMixin, merge_table_args, omop_index, ) +from omop_alchemy.cdm.model.clinical.event_metadata import ( + CLINICAL_EVENT_TARGETS_BY_FIELD_CONCEPT_ID, +) if TYPE_CHECKING: from ..vocabulary import Concept from .episode import Episode -@cache -def _modifier_target_classes_by_field_concept_id() -> dict[int, Type[Any]]: - """ - Default field-concept -> ORM target class map, CDM classes only. - - Scans every registered mapper, so iteration order is otherwise at the - mercy of import order. Sorting makes the result deterministic, and - restricting to ``cdm.model`` (the same ``MODEL_MODULE_PREFIX`` that - ``@cdm_table`` itself classifies tables by) excludes toolkit/domain - subclasses (e.g. oncology-aware views) that also implement - ``ModifierTargetMixin`` -- those are reached only through an explicit - override such as ``OncologyEpisodeEvent.resolved_event_target_classes``, - never by accident here. - """ - classes: dict[int, Type[Any]] = {} - mappers = sorted( - Base.registry.mappers, - key=lambda mapper: f"{mapper.class_.__module__}.{mapper.class_.__qualname__}", - ) - for mapper in mappers: - cls = mapper.class_ - if not issubclass(cls, ModifierTargetMixin): - continue - if not cls.__module__.startswith(MODEL_MODULE_PREFIX): - continue - try: - field_concept_id = cls.modifier_field_concept_id() - except NotImplementedError: - continue - if field_concept_id not in classes: - classes[field_concept_id] = cls - return classes - -@event.listens_for(Mapper, "after_mapper_constructed") -def _invalidate_target_class_cache( - mapper: Mapper[Any], - class_: type[Any], -) -> None: - _modifier_target_classes_by_field_concept_id.cache_clear() - def clear_episode_event_target_class_cache() -> None: - """ - Clear cached episode_event field-concept target mappings. - - Invalidation is automatic: ``after_mapper_constructed`` clears the cache - whenever a new mapper is configured, so a ``ModifierTargetMixin`` subclass - imported late is picked up without intervention. This remains as an escape - hatch for callers that build target classes outside the mapper lifecycle. - """ - _modifier_target_classes_by_field_concept_id.cache_clear() + """Retain the former cache hook; stable event metadata needs no invalidation.""" @cdm_table @@ -82,16 +33,28 @@ class Episode_Event(CDMTableBase, Base): omop_index(__tablename__, "episode_event_field_concept_id"), ) - episode_id: so.Mapped[int] = so.mapped_column(sa.ForeignKey("episode.episode_id"),nullable=False,primary_key=True) - event_id: so.Mapped[int] = so.mapped_column(nullable=False,primary_key=True) - episode_event_field_concept_id: so.Mapped[int] = so.mapped_column(sa.ForeignKey("concept.concept_id"),nullable=False,primary_key=True) + episode_id: so.Mapped[int] = so.mapped_column( + sa.ForeignKey("episode.episode_id"), nullable=False, primary_key=True + ) + event_id: so.Mapped[int] = so.mapped_column(nullable=False, primary_key=True) + episode_event_field_concept_id: so.Mapped[int] = so.mapped_column( + sa.ForeignKey("concept.concept_id"), nullable=False, primary_key=True + ) def __repr__(self) -> str: return f"" - + + class Episode_EventContext(ReferenceContext): - episode: so.Mapped["Episode"] = ReferenceContext._reference_relationship(target="Episode",local_fk="episode_id",remote_pk="episode_id",) # type: ignore[assignment] - event_field: so.Mapped["Concept"] = ReferenceContext._reference_relationship(target="Concept",local_fk="episode_event_field_concept_id",remote_pk="concept_id",) # type: ignore[assignment] + episode: so.Mapped["Episode"] = ReferenceContext._reference_relationship( + target="Episode", + local_fk="episode_id", + ) # type: ignore[assignment] + event_field: so.Mapped["Concept"] = ReferenceContext._reference_relationship( + target="Concept", + local_fk="episode_event_field_concept_id", + ) # type: ignore[assignment] + class Episode_EventView(Episode_Event, Episode_EventContext, DomainValidationMixin): """ @@ -112,8 +75,11 @@ class Episode_EventView(Episode_Event, Episode_EventContext, DomainValidationMix def resolved_event_target_classes(cls) -> dict[int, Type[Any]]: """ Map episode_event field concepts to ORM classes that can receive them. + + A fresh mapping lets domain-specific subclasses override selected + targets without mutating the stable Core registry. """ - return _modifier_target_classes_by_field_concept_id() + return dict(CLINICAL_EVENT_TARGETS_BY_FIELD_CONCEPT_ID) @property def event_table(self) -> str | None: @@ -130,8 +96,13 @@ def resolved_event_class(self) -> Type[Any] | None: @cached_property def resolved_event(self) -> Any | None: """ - Resolve EVENT_ID to concrete OMOP row. - Cached per-instance. + Navigate one link to its concrete OMOP row, cached per instance. + + This existing convenience is deliberately session-bound: it returns + None without an attached session or known target class, and session.get + may load a target individually. It does not validate person identity. + Measurement/Observation intentionally have no equivalent implicit + lookup; use the bulk target/attachment queries for validated links. """ session = so.object_session(self) cls = self.resolved_event_class @@ -148,24 +119,21 @@ def __repr__(self): f"{target.__class__.__name__}#{self.event_id}>" ) return f"" - + @property def episode_start_datetime(self): - return ( - self.episode.episode_start_datetime - if self.episode else None - ) - + return self.episode.episode_start_datetime if self.episode else None + @property def resolved_event_id_column(self) -> str | None: """ Name of the ID column on the resolved event table. Derived from episode_event_field_concept_id metadata. - + Example: 'condition_occurrence.condition_occurrence_id' resolves to 'condition_occurrence_id' """ if self.event_field and "." in self.event_field.concept_name: return self.event_field.concept_name.split(".", 1)[1] - return None \ No newline at end of file + return None diff --git a/omop_alchemy/cdm/model/unstructured/note.py b/omop_alchemy/cdm/model/unstructured/note.py index 369f97c..48e9868 100644 --- a/omop_alchemy/cdm/model/unstructured/note.py +++ b/omop_alchemy/cdm/model/unstructured/note.py @@ -54,44 +54,37 @@ class NoteContext(ReferenceContext): person: so.Mapped["Person"] = ReferenceContext._reference_relationship( target="Person", local_fk="person_id", - remote_pk="person_id", ) # type: ignore[assignment] note_type_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship( target="Concept", local_fk="note_type_concept_id", - remote_pk="concept_id", ) # type: ignore[assignment] note_class_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship( target="Concept", local_fk="note_class_concept_id", - remote_pk="concept_id", ) # type: ignore[assignment] encoding_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship( target="Concept", local_fk="encoding_concept_id", - remote_pk="concept_id", ) # type: ignore[assignment] language_concept: so.Mapped["Concept"] = ReferenceContext._reference_relationship( target="Concept", local_fk="language_concept_id", - remote_pk="concept_id", ) # type: ignore[assignment] provider: so.Mapped[Optional["Provider"]] = ReferenceContext._reference_relationship( target="Provider", local_fk="provider_id", - remote_pk="provider_id", ) # type: ignore[assignment] visit_occurrence: so.Mapped[Optional["Visit_Occurrence"]] = ( ReferenceContext._reference_relationship( target="Visit_Occurrence", local_fk="visit_occurrence_id", - remote_pk="visit_occurrence_id", ) ) # type: ignore[assignment] @@ -99,7 +92,6 @@ class NoteContext(ReferenceContext): ReferenceContext._reference_relationship( target="Visit_Detail", local_fk="visit_detail_id", - remote_pk="visit_detail_id", ) ) # type: ignore[assignment] @@ -107,7 +99,6 @@ class NoteContext(ReferenceContext): ReferenceContext._reference_relationship( target="Concept", local_fk="note_event_field_concept_id", - remote_pk="concept_id", ) ) # type: ignore[assignment] diff --git a/omop_alchemy/cdm/model/vocabulary/concept.py b/omop_alchemy/cdm/model/vocabulary/concept.py index 5bab7ec..89af3e0 100644 --- a/omop_alchemy/cdm/model/vocabulary/concept.py +++ b/omop_alchemy/cdm/model/vocabulary/concept.py @@ -104,9 +104,9 @@ class ConceptContext(ReferenceContext): foreign keys into reference tables and hierarchy navigation. """ - domain: so.Mapped["Domain"] = ReferenceContext._reference_relationship(target="Domain",local_fk="domain_id",remote_pk="domain_id") # type: ignore[assignment] - vocabulary: so.Mapped["Vocabulary"] = ReferenceContext._reference_relationship(target="Vocabulary",local_fk="vocabulary_id",remote_pk="vocabulary_id") # type: ignore[assignment] - concept_class: so.Mapped["Concept_Class"] = ReferenceContext._reference_relationship(target="Concept_Class",local_fk="concept_class_id",remote_pk="concept_class_id") # type: ignore[assignment] + domain: so.Mapped["Domain"] = ReferenceContext._reference_relationship(target="Domain",local_fk="domain_id") # type: ignore[assignment] + vocabulary: so.Mapped["Vocabulary"] = ReferenceContext._reference_relationship(target="Vocabulary",local_fk="vocabulary_id") # type: ignore[assignment] + concept_class: so.Mapped["Concept_Class"] = ReferenceContext._reference_relationship(target="Concept_Class",local_fk="concept_class_id") # type: ignore[assignment] @declared_attr def outgoing_relationships(cls) -> so.Mapped[List["Concept_Relationship"]]: diff --git a/omop_alchemy/cdm/query.py b/omop_alchemy/cdm/query.py index 724e705..e878aba 100644 --- a/omop_alchemy/cdm/query.py +++ b/omop_alchemy/cdm/query.py @@ -1,7 +1,4 @@ -"""Shared CDM concept-table query filtering. - -Consolidates filtering logic previously duplicated across downstream packages -""" +"""Shared CDM concept-table query filtering.""" from __future__ import annotations @@ -59,7 +56,7 @@ def __post_init__(self) -> None: ) def apply(self, query: sa.Select) -> sa.Select: - """Apply filter constraints to a Select already targeting Concept.""" + """Apply filter constraints to a Select whose FROM clause includes Concept.""" if self.concept_ids is not None: query = query.where(Concept.concept_id.in_(self.concept_ids)) diff --git a/omop_alchemy/toolkit/_utils.py b/omop_alchemy/toolkit/_utils.py new file mode 100644 index 0000000..173ef84 --- /dev/null +++ b/omop_alchemy/toolkit/_utils.py @@ -0,0 +1,60 @@ +"""Shared implementation details for toolkit query builders.""" + +from __future__ import annotations + +from collections.abc import Collection, Iterable, Sequence +from typing import Any + +import sqlalchemy as sa + +from sqlalchemy.sql.selectable import FromClause, SelectBase + + +def _nullable_column( + model: type[Any], + name: str, + sql_type: sa.types.TypeEngine[Any], +) -> sa.ColumnElement[Any]: + """Project a labelled column or a typed NULL when the model lacks it.""" + column = getattr(model, str(name), None) + if column is None: + # Unions need the same column positions across heterogeneous sources. + return sa.cast(sa.null(), sql_type).label(str(name)) + return column.label(str(name)) + + +def _select_or_union_all( + projections: Sequence[sa.Select[Any]], + *, + error_message: str, +) -> sa.Select[Any] | sa.CompoundSelect[Any]: + """Return one projection directly or combine several with ``UNION ALL``.""" + if not projections: + raise ValueError(error_message) + return projections[0] if len(projections) == 1 else sa.union_all(*projections) + + +def _as_from_clause( + source: FromClause | SelectBase, + *, + name: str, +) -> FromClause: + """Coerce a selectable to a named ``FromClause`` for query composition.""" + if isinstance(source, SelectBase): + return source.subquery(name) + if isinstance(source, FromClause): + return source + raise TypeError(f"{name} must be a SQLAlchemy Select or FromClause") + + +def _require_columns( + available: Collection[str], + required: Iterable[str], + *, + role: str, + error_type: type[Exception], +) -> None: + """Raise a domain-specific error when required column names are absent.""" + missing = tuple(sorted(set(required) - set(available))) + if missing: + raise error_type(f"{role} is missing required columns: {', '.join(missing)}") diff --git a/omop_alchemy/toolkit/analytics/body_metrics/weight_trajectory.py b/omop_alchemy/toolkit/analytics/body_metrics/weight_trajectory.py index eb2ffd9..b5794ab 100644 --- a/omop_alchemy/toolkit/analytics/body_metrics/weight_trajectory.py +++ b/omop_alchemy/toolkit/analytics/body_metrics/weight_trajectory.py @@ -65,6 +65,7 @@ def normalize_weight_readings( readings: list[MeasurementReading], rules: BodyMetricRules, ) -> list[MeasurementReading]: + """Convert supported weight units to kilograms and drop invalid values.""" normalized: list[MeasurementReading] = [] for reading in readings: kg = rules.normalize_weight_kg(reading.value, reading.unit_concept_id) @@ -78,6 +79,7 @@ def normalize_height_readings( readings: list[MeasurementReading], rules: BodyMetricRules, ) -> list[MeasurementReading]: + """Convert supported height units to centimetres and drop invalid values.""" normalized: list[MeasurementReading] = [] for reading in readings: cm = rules.normalize_height_cm(reading.value, reading.unit_concept_id) @@ -125,15 +127,20 @@ def _raw_height_series(self) -> list[MeasurementReading]: @cached_property def weight_readings(self) -> list[MeasurementReading]: """Weight readings normalized to kg.""" - return normalize_weight_readings(self._raw_weight_series, self.body_metric_rules()) + return normalize_weight_readings( + self._raw_weight_series, self.body_metric_rules() + ) @cached_property def height_readings_cm(self) -> list[MeasurementReading]: """Height readings normalized to cm and resolved without an episode date window.""" - return normalize_height_readings(self._raw_height_series, self.body_metric_rules()) + return normalize_height_readings( + self._raw_height_series, self.body_metric_rules() + ) @property def height_m(self) -> Optional[float]: + """Return the first normalized height as metres, when available.""" readings = self.height_readings_cm if not readings or readings[0].value is None: return None @@ -141,14 +148,17 @@ def height_m(self) -> Optional[float]: @property def baseline_weight(self) -> Optional[MeasurementReading]: + """Return the earliest normalized weight reading in the series.""" return self.weight_readings[0] if self.weight_readings else None @property def latest_weight(self) -> Optional[MeasurementReading]: + """Return the latest normalized weight reading in the series.""" return self.weight_readings[-1] if self.weight_readings else None @property def baseline_bmi(self) -> Optional[float]: + """Calculate BMI from the baseline weight and first available height.""" baseline = self.baseline_weight return self.body_metric_rules().bmi( baseline.value if baseline else None, @@ -157,6 +167,7 @@ def baseline_bmi(self) -> Optional[float]: @property def baseline_bsa_mosteller_m2(self) -> Optional[float]: + """Calculate Mosteller BSA from baseline weight and first height.""" baseline = self.baseline_weight height = self.height_readings_cm[0] if self.height_readings_cm else None return self.body_metric_rules().bsa_mosteller_m2( @@ -168,6 +179,7 @@ def pct_change_from_baseline( self, as_of: Optional[MeasurementReading] = None, ) -> WeightChange: + """Compare a target reading with baseline, preserving non-evaluability.""" baseline = self.baseline_weight target = as_of or self.latest_weight if ( @@ -183,6 +195,7 @@ def pct_change_from_baseline( return WeightChange(pct_change=pct, evaluable=True, reference=baseline) def pct_change_over(self, days: int) -> WeightChange: + """Compare latest weight with the earliest reading in the time window.""" readings = self.weight_readings if len(readings) < 2: return WeightChange.not_evaluable() @@ -197,10 +210,15 @@ def pct_change_over(self, days: int) -> WeightChange: or earliest_in_window.measurement_id == latest.measurement_id ): return WeightChange.not_evaluable() - pct = 100.0 * (latest.value - earliest_in_window.value) / earliest_in_window.value - return WeightChange(pct_change=pct, evaluable=True, reference=earliest_in_window) + pct = ( + 100.0 * (latest.value - earliest_in_window.value) / earliest_in_window.value + ) + return WeightChange( + pct_change=pct, evaluable=True, reference=earliest_in_window + ) def pct_change_trajectory(self) -> list[WeightTrajectoryPoint]: + """Return each valid reading's percentage change from baseline.""" baseline = self.baseline_weight if baseline is None or baseline.value is None or baseline.value <= 0: return [] @@ -225,8 +243,10 @@ def sustained_loss( threshold_pct: float = 5.0, min_consecutive: int = 2, ) -> Optional[bool]: + """Test whether recent readings sustain the requested loss threshold.""" baseline = self.baseline_weight - # the baseline is the first reading, so it cannot count towards a loss against itself + # Baseline is the first reading, so it cannot count toward a loss + # against itself; insufficient post-baseline data remains unknown. post_baseline = self.weight_readings[1:] if ( baseline is None @@ -245,6 +265,7 @@ def sustained_loss( ) def weight_trajectory_summary(self) -> WeightTrajectorySummary: + """Return the stable summary contract used by analytics consumers.""" baseline = self.baseline_weight latest = self.latest_weight change_from_baseline = self.pct_change_from_baseline() diff --git a/omop_alchemy/toolkit/analytics/oncology/__init__.py b/omop_alchemy/toolkit/analytics/oncology/__init__.py index 8a8e7de..e3b81a3 100644 --- a/omop_alchemy/toolkit/analytics/oncology/__init__.py +++ b/omop_alchemy/toolkit/analytics/oncology/__init__.py @@ -3,7 +3,14 @@ DIAGNOSTIC_STAGING_PROCEDURES, RADIOTHERAPY_PROCEDURES, SACT_DRUGS, + GROUP_STAGE_CONCEPTS, + M_STAGE_CONCEPTS, + METASTATIC_DISEASE_CONCEPTS, + N_STAGE_CONCEPTS, + T_STAGE_CONCEPTS, + TUMOR_GRADE_CONCEPTS, disease_episode_type_concept_ids, + laterality_modifier_concept_id, overarching_episode_type_concept_id, resolve_cancer_indicating_surgery_procedure_concept_ids, resolve_diagnostic_staging_procedure_concept_ids, @@ -12,6 +19,15 @@ treatment_cycle_episode_concept_id, treatment_episode_type_concept_ids, treatment_regimen_episode_concept_id, + tumor_size_modifier_concept_id, +) +from .condition_modifiers import ( + DEFAULT_STAGE_SELECTION, + StageBasis, + StageSelectionSpec, + preferred_stage_select, + stage_basis_expression, + stage_basis_priority_expression, ) from .oncology_critical_weight_loss import ( CriticalWeightLossSummary, @@ -39,8 +55,17 @@ "CANCER_INDICATING_SURGERY", "CriticalWeightLossSummary", "DIAGNOSTIC_STAGING_PROCEDURES", + "DEFAULT_STAGE_SELECTION", + "GROUP_STAGE_CONCEPTS", + "M_STAGE_CONCEPTS", + "METASTATIC_DISEASE_CONCEPTS", + "N_STAGE_CONCEPTS", "RADIOTHERAPY_PROCEDURES", "SACT_DRUGS", + "StageBasis", + "StageSelectionSpec", + "T_STAGE_CONCEPTS", + "TUMOR_GRADE_CONCEPTS", "OncologyCriticalWeightLossMixin", "OncologyDrugExposure", "OncologyEpisode", @@ -53,7 +78,9 @@ "RTDoseSummary", "SACTDoseSummary", "disease_episode_type_concept_ids", + "laterality_modifier_concept_id", "overarching_episode_type_concept_id", + "preferred_stage_select", "resolve_cancer_indicating_surgery_procedure_concept_ids", "resolve_diagnostic_staging_procedure_concept_ids", "resolve_rt_procedure_concept_ids", @@ -63,7 +90,10 @@ "sact_dose_evaluability", "summarize_rt_procedures_by", "summarize_sact_exposures_by", + "stage_basis_expression", + "stage_basis_priority_expression", "treatment_cycle_episode_concept_id", "treatment_episode_type_concept_ids", "treatment_regimen_episode_concept_id", + "tumor_size_modifier_concept_id", ] diff --git a/omop_alchemy/toolkit/analytics/oncology/concept_sets.py b/omop_alchemy/toolkit/analytics/oncology/concept_sets.py index 8405a65..1252d57 100644 --- a/omop_alchemy/toolkit/analytics/oncology/concept_sets.py +++ b/omop_alchemy/toolkit/analytics/oncology/concept_sets.py @@ -1,10 +1,7 @@ """Governed oncology concept sets. -Every set here names an omop-semantics semantic unit rather than assembling -concept IDs locally. That matters beyond tidiness: "what counts as -radiotherapy" is a clinical claim, and it was previously written out by hand -both here and in omop-constructs, governed by neither. omop-semantics 0.6.0 -publishes these as governed units, so both consumers name the same definition. +Every set here references a governed omop-semantics semantic unit. The unit +defines clinical membership, such as which concepts count as radiotherapy. Specs are declarative — importing this module resolves no semantics runtime and touches no database. Expansion happens on first use and is cached per @@ -16,70 +13,68 @@ from __future__ import annotations -from typing import Any - import sqlalchemy.orm as so from omop_alchemy.toolkit.core._semantics import default_semantics_runtime from omop_alchemy.toolkit.core.concepts import ( ConceptGroupSpec, ResolvedConceptGroup, + SemanticUnitRef, resolve_concept_group, ) -def _unit(value_set_name: str, unit_name: str) -> Any: - """Resolve a governed semantic unit, lazily. - - Deferred rather than captured at import so that declaring a spec does not - load the semantics runtime. - """ - return getattr(getattr(default_semantics_runtime(), value_set_name), unit_name) - - -class _LazyUnit: - """Attribute proxy that resolves its semantic unit on first access. - - ``ConceptGroupSpec`` reads ``parent_ids`` / ``excluded_parent_ids`` / - ``exact_ids`` off its ``unit``. Holding a proxy rather than the unit itself - keeps module import free of semantics loading, which is what allows basic - CDM work to avoid paying for oncology concept sets. - """ - - __slots__ = ("_value_set", "_unit") - - def __init__(self, value_set_name: str, unit_name: str) -> None: - self._value_set = value_set_name - self._unit = unit_name - - def __getattr__(self, name: str) -> Any: - return getattr(_unit(self._value_set, self._unit), name) - - def __repr__(self) -> str: - return f"" - - # Governed concept sets. Names are the governed semantic-unit names, which are # also the cache keys -- so the cache key derives from the governed identity # rather than a locally invented label. RADIOTHERAPY_PROCEDURES = ConceptGroupSpec( name="radiotherapy", - unit=_LazyUnit("cancer_procedures", "radiotherapy"), + unit=SemanticUnitRef("cancer_procedures", "radiotherapy"), ) CANCER_INDICATING_SURGERY = ConceptGroupSpec( name="cancer_indicating_surgery", - unit=_LazyUnit("cancer_procedures", "cancer_indicating_surgery"), + unit=SemanticUnitRef("cancer_procedures", "cancer_indicating_surgery"), ) DIAGNOSTIC_STAGING_PROCEDURES = ConceptGroupSpec( name="diagnostic_staging_procedure", - unit=_LazyUnit("cancer_procedures", "diagnostic_staging_procedure"), + unit=SemanticUnitRef("cancer_procedures", "diagnostic_staging_procedure"), ) SACT_DRUGS = ConceptGroupSpec( name="sact_drug_classification", - unit=_LazyUnit("sact", "sact_drug_classification"), + unit=SemanticUnitRef("sact", "sact_drug_classification"), +) + +T_STAGE_CONCEPTS = ConceptGroupSpec( + name="t_stage_concepts", + unit=SemanticUnitRef("staging", "t_stage_concepts"), +) + +N_STAGE_CONCEPTS = ConceptGroupSpec( + name="n_stage_concepts", + unit=SemanticUnitRef("staging", "n_stage_concepts"), +) + +M_STAGE_CONCEPTS = ConceptGroupSpec( + name="m_stage_concepts", + unit=SemanticUnitRef("staging", "m_stage_concepts"), +) + +GROUP_STAGE_CONCEPTS = ConceptGroupSpec( + name="group_stage_concepts", + unit=SemanticUnitRef("staging", "group_stage_concepts"), +) + +TUMOR_GRADE_CONCEPTS = ConceptGroupSpec( + name="tumor_grade", + unit=SemanticUnitRef("condition_modifiers", "tumor_grade"), +) + +METASTATIC_DISEASE_CONCEPTS = ConceptGroupSpec( + name="metastatic_disease_concepts", + unit=SemanticUnitRef("condition_modifiers", "metastatic_disease_concepts"), ) @@ -123,3 +118,17 @@ def treatment_regimen_episode_concept_id() -> int: def treatment_cycle_episode_concept_id() -> int: return default_semantics_runtime().types.treatment_episode_types.treatment_cycle + + +def laterality_modifier_concept_id() -> int: + """Governed modifier concept used when a value records laterality.""" + return int( + default_semantics_runtime().condition_modifiers.condition_modifier_values.laterality + ) + + +def tumor_size_modifier_concept_id() -> int: + """Governed numeric modifier concept used for tumour size.""" + return int( + default_semantics_runtime().condition_modifiers.numeric_condition_modifiers.tumor_size + ) diff --git a/omop_alchemy/toolkit/analytics/oncology/condition_modifiers.py b/omop_alchemy/toolkit/analytics/oncology/condition_modifiers.py new file mode 100644 index 0000000..60bd49c --- /dev/null +++ b/omop_alchemy/toolkit/analytics/oncology/condition_modifiers.py @@ -0,0 +1,150 @@ +"""Oncology policies for condition modifiers and preferred stage values.""" + +from __future__ import annotations + +from dataclasses import dataclass +from enum import StrEnum +from typing import Any + +import sqlalchemy as sa +from sqlalchemy.sql.selectable import FromClause, SelectBase + +from omop_alchemy.toolkit._utils import _as_from_clause +from omop_alchemy.toolkit.core.modifiers import ( + ModifierSelectionPolicy, + ModifierSelectionSpec, + selected_modifier_select, +) + + +class StageBasis(StrEnum): + pathological = "pathological" + clinical = "clinical" + unclassified = "unclassified" + + +@dataclass(frozen=True, slots=True) +class StageSelectionSpec: + """Preferred stage basis followed by temporal tie-breaking policy. + + The default prefers pathological stage, then clinical stage, then values + whose vocabulary code does not identify a basis. Supply an empty + ``basis_priority`` for chronological-only selection. + """ + + basis_priority: tuple[StageBasis, ...] = ( + StageBasis.pathological, + StageBasis.clinical, + StageBasis.unclassified, + ) + temporal_policy: ModifierSelectionPolicy = ModifierSelectionPolicy.earliest + + def __post_init__(self) -> None: + expected = set(StageBasis) + if self.basis_priority and ( + len(self.basis_priority) != len(expected) + or set(self.basis_priority) != expected + ): + raise ValueError( + "basis_priority must be empty or contain each StageBasis exactly once" + ) + + @classmethod + def clinical_first( + cls, + *, + temporal_policy: ModifierSelectionPolicy = ModifierSelectionPolicy.earliest, + ) -> StageSelectionSpec: + return cls( + basis_priority=( + StageBasis.clinical, + StageBasis.pathological, + StageBasis.unclassified, + ), + temporal_policy=temporal_policy, + ) + + @classmethod + def chronological_only( + cls, + *, + temporal_policy: ModifierSelectionPolicy = ModifierSelectionPolicy.earliest, + ) -> StageSelectionSpec: + return cls( + basis_priority=(), + temporal_policy=temporal_policy, + ) + + +DEFAULT_STAGE_SELECTION = StageSelectionSpec() + + +def stage_basis_expression( + concept_code: sa.ColumnElement[Any], +) -> sa.ColumnElement[str]: + """Classify stage codes using the source vocabulary's p/c convention. + + The staging vocabulary does not expose a reliable parent concept that + separates pathological from clinical staging. The code prefix is therefore + the intentional source-level contract rather than a synthetic hierarchy. + """ + normalized = sa.func.lower(sa.func.trim(concept_code)) + return sa.case( + (normalized.like("p%"), str(StageBasis.pathological)), + (normalized.like("c%"), str(StageBasis.clinical)), + else_=str(StageBasis.unclassified), + ) + + +def stage_basis_priority_expression( + concept_code: sa.ColumnElement[Any], + spec: StageSelectionSpec = DEFAULT_STAGE_SELECTION, +) -> sa.ColumnElement[int]: + """Render the configured stage-basis preference as a sortable SQL CASE.""" + if not spec.basis_priority: + return sa.literal(0) + basis = stage_basis_expression(concept_code) + return sa.case( + *( + (basis == str(candidate), rank) + for rank, candidate in enumerate(spec.basis_priority) + ), + else_=len(spec.basis_priority), + ) + + +def preferred_stage_select( + source: FromClause | SelectBase, + *, + spec: StageSelectionSpec = DEFAULT_STAGE_SELECTION, + concept_code_column: str | None = None, +) -> sa.Select[Any]: + """Select one preferred stage modifier for every canonical target. + + Basis-ranked selection requires a source enriched with a concept-code + column. Chronological-only selection does not require that enrichment. + """ + modifiers = _as_from_clause(source, name="preferred_stage_source") + priority: tuple[sa.ColumnElement[Any], ...] = () + if spec.basis_priority: + if concept_code_column is None: + raise ValueError( + "concept_code_column is required when basis ranking is enabled" + ) + if not concept_code_column.strip(): + raise ValueError("concept_code_column must not be empty") + if concept_code_column not in modifiers.c: + raise ValueError( + "stage source is missing required concept-code column: " + f"{concept_code_column}" + ) + priority = ( + stage_basis_priority_expression( + modifiers.c[concept_code_column], spec + ).asc(), + ) + return selected_modifier_select( + modifiers, + spec=ModifierSelectionSpec(policy=spec.temporal_policy), + priority=priority, + ) diff --git a/omop_alchemy/toolkit/core/__init__.py b/omop_alchemy/toolkit/core/__init__.py index 2a94bbe..0054745 100644 --- a/omop_alchemy/toolkit/core/__init__.py +++ b/omop_alchemy/toolkit/core/__init__.py @@ -9,6 +9,14 @@ Map free text and source codes to OMOP concept IDs, and hold the normalisation rules that make those mappings reproducible. +``events`` + Canonical cross-table event identities and projection row shapes shared by + timelines and episode builders. + +``modifiers`` + Canonical Measurement/Observation modifier rows, polymorphic target + validation, diagnostics, and deterministic selection. + ``timeline`` Project heterogeneous clinical rows into a single ordered sequence of events for one person. @@ -18,5 +26,7 @@ Nothing in core imports from ``episodes``, ``analytics``, or ``integrations``. Domain-specific concept sets, thresholds, and grading -rules belong with their domain under ``analytics``, not here. +rules belong with their domain under ``analytics``, not here. Generic database +lifecycle mechanics, including materialized-view creation and refresh, belong +to ``orm-loader``. """ diff --git a/omop_alchemy/toolkit/core/_ranking.py b/omop_alchemy/toolkit/core/_ranking.py new file mode 100644 index 0000000..c13d130 --- /dev/null +++ b/omop_alchemy/toolkit/core/_ranking.py @@ -0,0 +1,22 @@ +"""Shared internal construction for deterministic SQL window ranks.""" + +from __future__ import annotations + +from collections.abc import Iterable +from typing import Any + +import sqlalchemy as sa + + +def deterministic_row_number( + *, + partition_by: Iterable[sa.ColumnElement[Any]], + order_by: Iterable[sa.ColumnElement[Any]], + label: str, +) -> sa.ColumnElement[int]: + """Build a deterministic ``row_number`` expression for toolkit selectors.""" + return ( + sa.func.row_number() + .over(partition_by=tuple(partition_by), order_by=tuple(order_by)) + .label(label) + ) diff --git a/omop_alchemy/toolkit/core/concepts/__init__.py b/omop_alchemy/toolkit/core/concepts/__init__.py index 6880357..f80b60f 100644 --- a/omop_alchemy/toolkit/core/concepts/__init__.py +++ b/omop_alchemy/toolkit/core/concepts/__init__.py @@ -99,29 +99,52 @@ concept_group_registry, resolve_concept_group, ) +from .relationships import ( + STANDARD_CONCEPT_MAPPING_COLUMNS, + STANDARD_CONCEPT_MAPPING_UNIQUENESS, + StandardConceptMappingColumn, + StandardConceptMappingSpec, + standard_concept_mapping_select, +) +from .runtime import ( + RuntimeConceptSetSpec, + descendant_concept_select, + runtime_concept_predicate, +) +from .semantics import ConceptGroupAnchors, SemanticUnitRef __all__ = [ "DEFAULT_MAX_CACHE_BYTES", "CacheStats", + "STANDARD_CONCEPT_MAPPING_COLUMNS", + "STANDARD_CONCEPT_MAPPING_UNIQUENESS", "ConceptGroupRegistry", + "ConceptGroupAnchors", "ConceptGroupSpec", + "StandardConceptMappingColumn", + "StandardConceptMappingSpec", "ConceptResolver", "ConceptResolverRegistry", "LookupIndex", "LookupSpec", "OMOPConceptSource", "ResolvedConceptGroup", + "SemanticUnitRef", + "RuntimeConceptSetSpec", "build_concept_group", "clear_concept_group_cache", "clear_vocabulary_identity", "compose_normalizers", "concept_group_cache_stats", "concept_group_registry", + "standard_concept_mapping_select", "make_concept_resolver", "make_stage", "normalize_default", "register_vocabulary_identity", "resolve_concept_group", + "descendant_concept_select", + "runtime_concept_predicate", "site_to_NOS", "strip_uicc", ] diff --git a/omop_alchemy/toolkit/core/concepts/groups.py b/omop_alchemy/toolkit/core/concepts/groups.py index f3fed7a..ac1b9d7 100644 --- a/omop_alchemy/toolkit/core/concepts/groups.py +++ b/omop_alchemy/toolkit/core/concepts/groups.py @@ -23,12 +23,13 @@ from __future__ import annotations from dataclasses import dataclass, field -from typing import Any, Iterable +from typing import Any import sqlalchemy as sa import sqlalchemy.orm as so -from omop_alchemy.cdm.model import Concept_Ancestor +from .runtime import descendant_concept_select +from .semantics import ConceptGroupAnchors @dataclass(frozen=True) @@ -47,19 +48,17 @@ class ConceptGroupSpec: unit The omop-semantics ``RuntimeSemanticUnit`` supplying anchors. Read lazily, so a spec can be declared at module scope without loading the - semantics runtime. omop-semantics 0.6.0 put mixed-role composition on - the semantic unit rather than on ``RuntimeGroup``, which is why this - takes a unit: ``parent_ids`` expand through descendants while - ``exact_ids`` are matched directly. + semantics runtime. The unit supplies mixed-role anchors: + ``parent_ids`` expand through descendants while ``exact_ids`` are + matched directly. include_descendants Expand ``parent_ids`` through ``concept_ancestor``. When False only the anchors themselves are members. require_standard Restrict expansion to concepts carrying a standardness flag. Defaults to - False, matching the historical oncology behaviour of reading - ``concept_ancestor`` without a standard filter. Note this is the - *opposite* default to ``OMOPConceptSource.descendants``; the difference is - deliberate and declared here rather than left implicit. + False, so expansion reads ``concept_ancestor`` without a standard + filter. ``OMOPConceptSource.descendants`` defaults to requiring standard + concepts. include_classification Widens ``require_standard`` to admit classification ('C') concepts. Defaults to True so a governed group can be anchored on a classification @@ -68,7 +67,7 @@ class ConceptGroupSpec: """ name: str - unit: Any + unit: ConceptGroupAnchors include_descendants: bool = True require_standard: bool = False include_classification: bool = True @@ -107,7 +106,7 @@ def expression_for( if parents and self.include_descendants: expr = column.in_( - _descendant_select( + descendant_concept_select( parents, require_standard=self.require_standard, include_classification=self.include_classification, @@ -116,7 +115,7 @@ def expression_for( excluded = self.excluded_parent_ids() if excluded: expr = expr & column.not_in( - _descendant_select( + descendant_concept_select( excluded, require_standard=self.require_standard, include_classification=self.include_classification, @@ -134,30 +133,6 @@ def expression_for( return sa.or_(*clauses) -def _descendant_select( - parents: Iterable[int], - *, - require_standard: bool, - include_classification: bool = True, -) -> sa.Select: - stmt = sa.select(Concept_Ancestor.descendant_concept_id).where( - Concept_Ancestor.ancestor_concept_id.in_(tuple(parents)) - ) - if require_standard: - from omop_alchemy.cdm.model.vocabulary import Concept - - standardness = ( - sa.or_(Concept.is_standard_expr(), Concept.is_classification_expr()) - if include_classification - else Concept.is_standard_expr() - ) - stmt = stmt.join( - Concept, - Concept.concept_id == Concept_Ancestor.descendant_concept_id, - ).where(standardness) - return stmt - - @dataclass(frozen=True) class ResolvedConceptGroup: """A governed group expanded against one vocabulary. @@ -217,7 +192,7 @@ def build_concept_group( if parents: if spec.include_descendants: - stmt = _descendant_select( + stmt = descendant_concept_select( parents, require_standard=spec.require_standard, include_classification=spec.include_classification, @@ -225,7 +200,7 @@ def build_concept_group( ids |= set(session.execute(stmt).scalars().all()) excluded = spec.excluded_parent_ids() if excluded: - stmt = _descendant_select( + stmt = descendant_concept_select( excluded, require_standard=spec.require_standard, include_classification=spec.include_classification, diff --git a/omop_alchemy/toolkit/core/concepts/lookup.py b/omop_alchemy/toolkit/core/concepts/lookup.py index 92e3e88..09ee2f0 100644 --- a/omop_alchemy/toolkit/core/concepts/lookup.py +++ b/omop_alchemy/toolkit/core/concepts/lookup.py @@ -1,3 +1,17 @@ +"""Build and resolve scoped text-to-OMOP concept lookup indexes. + +The database-facing source materialises a deliberately bounded vocabulary +selection once; the runtime resolver then performs only in-memory +normalisation and correction. Keeping those responsibilities separate is +important for bulk ETL, where a resolver must not reopen relationships or +expand vocabulary hierarchies per row. + +OMOP Alchemy's resolver is scoped deterministic lookup over that materialised +index. Candidate generation, graph traversal and grounding constraints belong +to omop-graph; the two layers may share vocabulary concepts without sharing a +resolver contract. +""" + from typing import Iterable, Callable from dataclasses import dataclass from functools import cached_property @@ -9,16 +23,9 @@ from omop_alchemy.cdm.model.vocabulary import Concept, Concept_Synonym, Concept_Ancestor from omop_alchemy.cdm.query import ConceptFilter -""" -Class definitions for vocabulary handling and mapping. - -This is somewhat redundant with some of omop-graph but -mutual dependency is awkward and this is a thin layer so -it's not worth over-engineering separation at this stage. -""" - Normaliser = Callable[[str], str] + @dataclass(frozen=True) class LookupIndex: """ @@ -43,24 +50,27 @@ class LookupIndex: Notes ----- The mapping may contain multiple textual representations pointing to the - same concept ID (e.g. name + code + synonym). + same concept ID (e.g. name + code + synonym). """ + name: str unknown: int | None mapping: dict[str, int] def lookup(self, term: str | None) -> int | None: + """Resolve an already-normalized key, returning the configured fallback.""" if term is None: term = "" return self.mapping.get(term, self.unknown) def __contains__(self, item: str | int) -> bool: + """Test membership by indexed key or by reachable concept ID.""" if isinstance(item, str): return item in self.mapping if isinstance(item, int): return item in self.mapping.values() return False - + def __repr__(self) -> str: return ( f" str: @property def all_concepts(self) -> set[int]: + """Return the concept IDs represented by this materialized index.""" return set(self.mapping.values()) - + @dataclass(frozen=True) class LookupSpec: @@ -92,12 +103,12 @@ class LookupSpec: Attributes ---------- name: - Stable identifier for this lookup specification. + Stable identifier for this lookup specification. unknown: Concept ID to return for unmatched terms. Set to None to preserve nulls, or to a sentinel concept ID to force closed-world behaviour. domain_id: - Optional OMOP domain filter + Optional OMOP domain filter concept_class_id: Optional list of OMOP concept_class_id values to restrict the lookup vocabulary_id: @@ -144,6 +155,7 @@ class LookupSpec: build-time (this spec) and runtime (ConceptResolver) to make lookup behaviour explicit and testable. """ + name: str unknown: int | None = 0 domain_id: str | None = None @@ -168,8 +180,8 @@ class OMOPConceptSource: and higher-level vocabulary indexing logic. - Used exclusively to builds a query based on provided parameters (adds - filter for each non-None parameter, and joins to Concept_Ancestor + Used exclusively to builds a query based on provided parameters (adds + filter for each non-None parameter, and joins to Concept_Ancestor if parents are specified). """ @@ -203,7 +215,7 @@ def fetch_synonyms( for r in rows if r.concept_synonym_name ] - + @staticmethod def fetch_concepts( session: so.Session, @@ -290,7 +302,7 @@ def fetch_concepts( ) for r in rows ] - + @staticmethod def descendants( session: so.Session, @@ -298,6 +310,7 @@ def descendants( *, include_non_standard: bool = False, ) -> list[int]: + """Return descendant IDs using the source's standardness policy.""" rows = OMOPConceptSource.fetch_concepts( session, parents=parents, @@ -305,13 +318,20 @@ def descendants( require_standard=not include_non_standard, ) return list({r.concept_id for r in rows}) - @staticmethod def build_lookup( session: so.Session, spec: LookupSpec, ) -> LookupIndex: + """Materialize one scoped lookup and its optional synonym keys. + + The returned index is intentionally detached from the session. If + multiple selected representations normalize to the same key, the + later materialized assignment wins; callers that need collision-free + semantics should narrow the ``LookupSpec`` rather than rely on row + ordering. + """ rows = OMOPConceptSource.fetch_concepts( session, domain_id=spec.domain_id, @@ -335,17 +355,14 @@ def build_lookup( m[spec.normalizer(r.concept_code)] = r.concept_id if spec.include_synonyms: - for cid, syn in OMOPConceptSource.fetch_synonyms( - session, concept_ids=ids - ): + for cid, syn in OMOPConceptSource.fetch_synonyms(session, concept_ids=ids): if syn: m[spec.normalizer(syn)] = cid return LookupIndex(name=spec.name, unknown=spec.unknown, mapping=m) - -class ConceptResolver: +class ConceptResolver: """ Runtime resolver for mapping free-text terms to OMOP concept IDs. @@ -399,6 +416,7 @@ class ConceptResolver: 123456 """ + def __init__( self, index: LookupIndex, @@ -406,11 +424,13 @@ def __init__( normalizer: Normaliser | None = None, corrections: list[Callable[[str], str]] | None = None, ): + """Bind a materialized index to runtime normalization and corrections.""" self.index = index self._normalizer = normalizer or normalize_default self._corrections = corrections or [] def lookup(self, term: str | None) -> int | None: + """Resolve a term, trying direct lookup before ordered corrections.""" if not term: return self.index.unknown @@ -428,11 +448,13 @@ def lookup(self, term: str | None) -> int | None: return self.index.unknown def lookup_exact(self, term: str | None) -> int | None: + """Resolve only the normalized input, bypassing correction functions.""" if not term: return self.index.unknown return self.index.mapping.get(self._normalizer(term), self.index.unknown) def __contains__(self, item: str | int) -> bool: + """Test corrected text membership or direct concept-ID membership.""" if isinstance(item, int): return item in self.all_concepts if isinstance(item, str): @@ -443,9 +465,8 @@ def __contains__(self, item: str | int) -> bool: def all_concepts(self) -> set[int]: """Every concept ID reachable through this resolver's index. - Cached: the index is fixed at construction, and callers legitimately - union several resolvers' sets, which previously rebuilt each one per - access. Returned by reference, so treat it as read-only. + Cached because the index is fixed at construction. Returned by + reference, so treat it as read-only. """ return set(self.index.mapping.values()) @@ -502,7 +523,7 @@ def make_concept_resolver( name: Stable identifier for this lookup specification, used in logging and debugging. unknown: - Concept ID to return for unmatched terms. Set to None to preserve nulls, or to + Concept ID to return for unmatched terms. Set to None to preserve nulls, or to a sentinel concept ID to force closed-world behaviour. domain_id: Optional OMOP domain filter for the concepts to include in the lookup. @@ -524,29 +545,29 @@ def make_concept_resolver( text can still be resolved forward through "Maps to" / "Concept replaced by", whereas filtering it out here discards the term entirely. code_filter: - Optional substring filter applied to concept_code (ILIKE-based). + Optional substring filter applied to concept_code (ILIKE-based). Useful for coarse scoping (e.g. AJCC-only codes). parents: - Optional list of ancestor concept IDs from which to expand the lookup + Optional list of ancestor concept IDs from which to expand the lookup via the Concept_Ancestor table. include_non_standard_descendants: - If True, includes non-standard concepts when expanding from parents. Has no + If True, includes non-standard concepts when expanding from parents. Has no effect if `parents` is None. include_synonyms: If True, include Concept_Synonym entries in the lookup keys. include: - Tuple of ConceptRow attribute names to index as keys (e.g. ("concept_name", + Tuple of ConceptRow attribute names to index as keys (e.g. ("concept_name", "concept_code")). This controls which textual fields become resolvable inputs. build_normalizer: - Normalisation function applied to all indexed strings at build time. This should - match (or be compatible with) the normaliser used at resolution time by + Normalisation function applied to all indexed strings at build time. This should + match (or be compatible with) the normaliser used at resolution time by ConceptResolver. runtime_normalizer: - Optional normalisation function applied to input terms at lookup time. Defaults to - ``normalize_default``. This should be compatible with the normaliser used when + Optional normalisation function applied to input terms at lookup time. Defaults to + ``normalize_default``. This should be compatible with the normaliser used when constructing the LookupIndex. corrections: - Optional ordered list of correction functions applied to the raw input term prior + Optional ordered list of correction functions applied to the raw input term prior to normalisation and lookup """ diff --git a/omop_alchemy/toolkit/core/concepts/relationships.py b/omop_alchemy/toolkit/core/concepts/relationships.py new file mode 100644 index 0000000..426344a --- /dev/null +++ b/omop_alchemy/toolkit/core/concepts/relationships.py @@ -0,0 +1,139 @@ +"""Queries for resolving OMOP concepts to their standard representations.""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import date +from enum import StrEnum +from typing import Any + +import sqlalchemy as sa +import sqlalchemy.orm as so + +from omop_alchemy.cdm.model.vocabulary import Concept, Concept_Relationship + + +class StandardConceptMappingColumn(StrEnum): + """Stable labels emitted by :func:`standard_concept_mapping_select`.""" + + source_concept_id = "source_concept_id" + source_vocabulary_id = "source_vocabulary_id" + source_concept_code = "source_concept_code" + source_concept_name = "source_concept_name" + standard_concept_id = "standard_concept_id" + standard_vocabulary_id = "standard_vocabulary_id" + standard_concept_code = "standard_concept_code" + standard_concept_name = "standard_concept_name" + relationship_valid_start_date = "relationship_valid_start_date" + relationship_valid_end_date = "relationship_valid_end_date" + + +STANDARD_CONCEPT_MAPPING_COLUMNS: tuple[StandardConceptMappingColumn, ...] = tuple( + StandardConceptMappingColumn +) +"""Columns exposed by a standard concept mapping query.""" + + +STANDARD_CONCEPT_MAPPING_UNIQUENESS: tuple[StandardConceptMappingColumn, ...] = ( + StandardConceptMappingColumn.source_concept_id, + StandardConceptMappingColumn.standard_concept_id, +) +"""Executable uniqueness key of a standard concept mapping result.""" + + +@dataclass(frozen=True, slots=True) +class StandardConceptMappingSpec: + """Select valid ``Maps to`` relationships for the requested source concepts. + + An empty ``source_concept_ids`` tuple selects every source. ``valid_on`` can + be supplied when a reproducible historical vocabulary view is required; it + applies to both the relationship and its standard target. Invalid + relationships and targets are always excluded. + """ + + source_concept_ids: tuple[int, ...] = () + valid_on: date | None = None + + def __post_init__(self) -> None: + object.__setattr__( + self, + "source_concept_ids", + tuple(sorted(set(self.source_concept_ids))), + ) + + +def standard_concept_mapping_select( + spec: StandardConceptMappingSpec, +) -> sa.Select[Any]: + """Select each valid, single-hop ``Maps to`` standard-concept mapping. + + Results remain relational because OMOP permits one source concept to map to + more than one standard concept. Standard-concept self-maps are returned as + ordinary rows; no recursive traversal is required. + """ + source = so.aliased(Concept, name="mapping_source_concept") + standard = so.aliased(Concept, name="mapping_standard_concept") + relationship = so.aliased( + Concept_Relationship, + name="standard_mapping_relationship", + ) + + # Separate aliases preserve both sides of the mapping in the result and + # prevent source predicates from accidentally being applied to the target. + statement = ( + sa.select( + source.concept_id.label( + str(StandardConceptMappingColumn.source_concept_id) + ), + source.vocabulary_id.label( + str(StandardConceptMappingColumn.source_vocabulary_id) + ), + source.concept_code.label( + str(StandardConceptMappingColumn.source_concept_code) + ), + source.concept_name.label( + str(StandardConceptMappingColumn.source_concept_name) + ), + standard.concept_id.label( + str(StandardConceptMappingColumn.standard_concept_id) + ), + standard.vocabulary_id.label( + str(StandardConceptMappingColumn.standard_vocabulary_id) + ), + standard.concept_code.label( + str(StandardConceptMappingColumn.standard_concept_code) + ), + standard.concept_name.label( + str(StandardConceptMappingColumn.standard_concept_name) + ), + relationship.valid_start_date.label( + str(StandardConceptMappingColumn.relationship_valid_start_date) + ), + relationship.valid_end_date.label( + str(StandardConceptMappingColumn.relationship_valid_end_date) + ), + ) + .select_from(source) + .join(relationship, relationship.concept_id_1 == source.concept_id) + .join(standard, standard.concept_id == relationship.concept_id_2) + .where( + relationship.relationship_id == "Maps to", + relationship.is_valid_expr(), + standard.is_standard_expr(), + standard.is_valid_expr(), + ) + ) + + if spec.source_concept_ids: + statement = statement.where(source.concept_id.in_(spec.source_concept_ids)) + if spec.valid_on is not None: + # A historical mapping is valid only when both the relationship and + # the target concept existed at the requested date. + valid_on = sa.literal(spec.valid_on) + statement = statement.where( + relationship.valid_start_date <= valid_on, + relationship.valid_end_date >= valid_on, + standard.valid_start_date <= valid_on, + standard.valid_end_date >= valid_on, + ) + return statement diff --git a/omop_alchemy/toolkit/core/concepts/runtime.py b/omop_alchemy/toolkit/core/concepts/runtime.py new file mode 100644 index 0000000..d86cb16 --- /dev/null +++ b/omop_alchemy/toolkit/core/concepts/runtime.py @@ -0,0 +1,153 @@ +"""Declarative runtime concept-set inputs for database-side predicates. + +``ConceptGroupSpec`` is the right contract for governed omop-semantics units. +``RuntimeConceptSetSpec`` complements it for IDs supplied by configuration at +runtime. It records intent without expanding vocabulary hierarchies or touching +a database; ``runtime_concept_predicate`` renders the corresponding SQL. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any, Iterable + +import sqlalchemy as sa + +from omop_alchemy.cdm.model.vocabulary import Concept, Concept_Ancestor +from omop_alchemy.cdm.query import ConceptFilter + + +def _normalise_concept_ids(values: Iterable[int]) -> tuple[int, ...]: + """Return stable, duplicate-free inputs without imposing vocabulary policy.""" + return tuple(sorted(set(values))) + + +@dataclass(frozen=True, slots=True) +class RuntimeConceptSetSpec: + """Runtime include/exclude inputs for a database-side concept predicate. + + The intended expression is:: + + (included ancestor descendants OR included exact IDs) + AND NOT (excluded ancestor descendants OR excluded exact IDs) + + Exclusion therefore wins if the same concept is reached by both sides. + Empty inclusions describe an always-false set. Constructing the spec is + side-effect free and preserves no session-bound vocabulary objects. + + ``require_standard`` and ``include_classification`` deliberately match + ``ConceptFilter`` and ``ConceptGroupSpec`` for descendant expansion. + Exact IDs remain explicit inclusions even when the deployed vocabulary has + de-standardised one of them; configuration validation may report that drift + without silently changing set membership. Descendant rendering delegates to + the existing normalised ``Concept`` flag expressions. + + IDs are sorted and deduplicated only. Validity rules for configuration or a + local vocabulary belong at those boundaries, not in this generic spec. + """ + + include_ancestor_ids: tuple[int, ...] = () + include_exact_ids: tuple[int, ...] = () + exclude_ancestor_ids: tuple[int, ...] = () + exclude_exact_ids: tuple[int, ...] = () + require_standard: bool = False + include_classification: bool = True + + def __post_init__(self) -> None: + for field_name in ( + "include_ancestor_ids", + "include_exact_ids", + "exclude_ancestor_ids", + "exclude_exact_ids", + ): + object.__setattr__( + self, + field_name, + _normalise_concept_ids(getattr(self, field_name)), + ) + + @property + def has_inclusions(self) -> bool: + """Whether the predicate can match at least one configured input.""" + return bool(self.include_ancestor_ids or self.include_exact_ids) + + +def descendant_concept_select( + ancestor_ids: Iterable[int], + *, + require_standard: bool = False, + include_classification: bool = True, +) -> sa.Select[Any]: + """Select each descendant ID once for the supplied ancestors.""" + statement = ( + sa.select(Concept_Ancestor.descendant_concept_id) + .where(Concept_Ancestor.ancestor_concept_id.in_(tuple(ancestor_ids))) + .distinct() + ) + if not require_standard: + # Exact descendant expansion can remain a cheap ancestor-table query; + # join the concept table only when OMOP standardness is requested. + return statement + + statement = statement.join( + Concept, + Concept.concept_id == Concept_Ancestor.descendant_concept_id, + ) + return ConceptFilter( + require_standard=True, + include_classification=include_classification, + ).apply(statement) + + +def _concept_set_side( + column: sa.SQLColumnExpression[Any], + *, + ancestor_ids: tuple[int, ...], + exact_ids: tuple[int, ...], + require_standard: bool, + include_classification: bool, +) -> sa.ColumnElement[bool]: + clauses: list[sa.ColumnElement[bool]] = [] + if ancestor_ids: + clauses.append( + column.in_( + descendant_concept_select( + ancestor_ids, + require_standard=require_standard, + include_classification=include_classification, + ) + ) + ) + if exact_ids: + # Exact IDs are explicit configuration and intentionally bypass the + # descendant standardness filter; validation belongs at the config edge. + clauses.append(column.in_(exact_ids)) + return sa.or_(*clauses) if clauses else sa.false() + + +def runtime_concept_predicate( + column: sa.SQLColumnExpression[Any], + spec: RuntimeConceptSetSpec, +) -> sa.ColumnElement[bool]: + """Render runtime concept membership entirely as database predicates.""" + if not spec.has_inclusions: + return sa.false() + + included = _concept_set_side( + column, + ancestor_ids=spec.include_ancestor_ids, + exact_ids=spec.include_exact_ids, + require_standard=spec.require_standard, + include_classification=spec.include_classification, + ) + + excluded = _concept_set_side( + column, + ancestor_ids=spec.exclude_ancestor_ids, + exact_ids=spec.exclude_exact_ids, + require_standard=spec.require_standard, + include_classification=spec.include_classification, + ) + # Exclusion is evaluated after inclusion so an explicit exclusion always + # wins over a descendant or exact inclusion. + return sa.and_(included, sa.not_(excluded)) diff --git a/omop_alchemy/toolkit/core/concepts/semantics.py b/omop_alchemy/toolkit/core/concepts/semantics.py new file mode 100644 index 0000000..793407a --- /dev/null +++ b/omop_alchemy/toolkit/core/concepts/semantics.py @@ -0,0 +1,67 @@ +"""Deferred references to governed ``omop-semantics`` units. + +Domain packages declare concept groups as module-level constants. Resolving the +bundled semantics runtime while those modules import would make an optional +dependency mandatory and could make otherwise declarative imports perform +unwanted work. ``SemanticUnitRef`` stores only a stable path at construction and +resolves that path when a concept-group role is first read. + +The reference exposes all three roles of the governed unit without interpreting +or reshaping them. Narrow clinical groupings therefore remain governed in +``omop-semantics`` rather than being assembled locally by consumers. +""" + +from __future__ import annotations + +from collections.abc import Collection +from dataclasses import dataclass +from typing import Protocol + +from omop_alchemy.toolkit.core._semantics import default_semantics_runtime + + +class ConceptGroupAnchors(Protocol): + """Role-aware anchors consumed by :class:`ConceptGroupSpec`.""" + + @property + def parent_ids(self) -> Collection[int]: ... + + @property + def excluded_parent_ids(self) -> Collection[int]: ... + + @property + def exact_ids(self) -> Collection[int]: ... + + +@dataclass(frozen=True, slots=True) +class SemanticUnitRef: + """Lazy, immutable reference to one complete governed semantic unit.""" + + value_set: str + unit: str + + def __post_init__(self) -> None: + if not self.value_set.strip() or not self.unit.strip(): + raise ValueError("semantic value-set and unit names must not be empty") + + def _semantic_unit(self) -> object: + value_set = getattr(default_semantics_runtime(), self.value_set) + return getattr(value_set, self.unit) + + def _role_ids(self, role: str) -> frozenset[int]: + return frozenset(int(value) for value in getattr(self._semantic_unit(), role)) + + @property + def parent_ids(self) -> frozenset[int]: + return self._role_ids("parent_ids") + + @property + def excluded_parent_ids(self) -> frozenset[int]: + return self._role_ids("excluded_parent_ids") + + @property + def exact_ids(self) -> frozenset[int]: + return self._role_ids("exact_ids") + + def __repr__(self) -> str: + return f"" diff --git a/omop_alchemy/toolkit/core/events/__init__.py b/omop_alchemy/toolkit/core/events/__init__.py new file mode 100644 index 0000000..28fefcf --- /dev/null +++ b/omop_alchemy/toolkit/core/events/__init__.py @@ -0,0 +1,27 @@ +"""Canonical, domain-neutral clinical-event identities and row shapes. + +Event tables use different native column names, but cross-table analytical +queries need one stable vocabulary. This area provides both the shared row +contracts and SQLAlchemy projections. Building a projection is side-effect free; +the database is accessed only when a caller executes the returned statement. +""" + +from .contracts import ( + ClinicalEventColumn, + ClinicalEventIdentity, + ClinicalEventRow, + ValuedClinicalEventRow, +) +from .projections import ( + canonical_event_projection, + canonical_event_union, +) + +__all__ = [ + "ClinicalEventColumn", + "ClinicalEventIdentity", + "ClinicalEventRow", + "ValuedClinicalEventRow", + "canonical_event_projection", + "canonical_event_union", +] diff --git a/omop_alchemy/toolkit/core/events/contracts.py b/omop_alchemy/toolkit/core/events/contracts.py new file mode 100644 index 0000000..b5e15f5 --- /dev/null +++ b/omop_alchemy/toolkit/core/events/contracts.py @@ -0,0 +1,77 @@ +"""Side-effect-free contracts for canonical cross-table clinical events.""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import date, datetime +from enum import StrEnum +from typing import Protocol, runtime_checkable + + +class ClinicalEventColumn(StrEnum): + """Canonical labels emitted by a cross-table clinical-event projection.""" + + person_id = "person_id" + event_id = "event_id" + event_date = "event_date" + event_datetime = "event_datetime" + event_concept_id = "event_concept_id" + event_field_concept_id = "event_field_concept_id" + event_source_table = "event_source_table" + value_as_number = "value_as_number" + value_as_concept_id = "value_as_concept_id" + unit_concept_id = "unit_concept_id" + + @classmethod + def required_columns(cls) -> tuple[ClinicalEventColumn, ...]: + """Required projection labels in row-contract order.""" + return tuple(cls[name] for name in ClinicalEventRow.__annotations__) + + @classmethod + def optional_columns(cls) -> tuple[ClinicalEventColumn, ...]: + """Nullable value labels, excluding inherited required fields.""" + return tuple(cls[name] for name in ValuedClinicalEventRow.__annotations__) + + +@runtime_checkable +class ClinicalEventRow(Protocol): + """Value-level view of the required canonical event projection. + + SQLAlchemy ``Row`` objects and small dataclasses can both satisfy this + protocol. It describes the output consumed by downstream tools; it does not + require a session-bound ORM entity. + """ + + person_id: int + event_id: int + event_date: date + event_datetime: datetime | None + event_concept_id: int + event_field_concept_id: int + event_source_table: str + + +@runtime_checkable +class ValuedClinicalEventRow(ClinicalEventRow, Protocol): + """Canonical event row extended with nullable value and unit fields.""" + + value_as_number: float | None + value_as_concept_id: int | None + unit_concept_id: int | None + + +@dataclass(frozen=True, order=True, slots=True) +class ClinicalEventIdentity: + """Cross-table event identity. + + OMOP event IDs are unique only within their source table. A Measurement and + a Procedure Occurrence may legitimately have the same numeric ID, so the + table is a mandatory part of identity. + """ + + event_source_table: str + event_id: int + + def __post_init__(self) -> None: + if not self.event_source_table.strip(): + raise ValueError("event_source_table must not be empty") diff --git a/omop_alchemy/toolkit/core/events/projections.py b/omop_alchemy/toolkit/core/events/projections.py new file mode 100644 index 0000000..e2fc522 --- /dev/null +++ b/omop_alchemy/toolkit/core/events/projections.py @@ -0,0 +1,82 @@ +"""SQLAlchemy projections for a consistent cross-table clinical-event shape.""" + +from __future__ import annotations + +from typing import Any + +import sqlalchemy as sa + +from omop_alchemy.cdm.model.clinical.event_metadata import ( + clinical_event_model_spec as _clinical_event_model_spec, +) +from omop_alchemy.toolkit._utils import _nullable_column, _select_or_union_all + +from .contracts import ClinicalEventColumn + + +def canonical_event_projection( + model: type[Any], + *, + include_values: bool = True, +) -> sa.Select[Any]: + """Project one supported OMOP event model to canonical event columns.""" + spec = _clinical_event_model_spec(model) + # The output deliberately uses canonical labels rather than source names; + # downstream attachment, timeline, and union code should not branch on the + # particular OMOP event table being projected. + event_datetime = ( + getattr(model, spec.event_datetime_column) + if spec.event_datetime_column is not None + else sa.cast(sa.null(), sa.DateTime) + ) + columns: list[sa.ColumnElement[Any]] = [ + model.person_id.label(str(ClinicalEventColumn.person_id)), + getattr(model, spec.event_id_column).label(str(ClinicalEventColumn.event_id)), + getattr(model, spec.event_date_column).label( + str(ClinicalEventColumn.event_date) + ), + event_datetime.label(str(ClinicalEventColumn.event_datetime)), + getattr(model, spec.event_concept_id_column).label( + str(ClinicalEventColumn.event_concept_id) + ), + sa.literal(spec.event_field_concept_id).label( + str(ClinicalEventColumn.event_field_concept_id) + ), + sa.literal(spec.event_source_table).label( + str(ClinicalEventColumn.event_source_table) + ), + ] + if include_values: + # Value fields are optional but occupy fixed positions when requested, + # allowing heterogeneous event projections to be combined with UNION ALL. + columns.extend( + ( + _nullable_column( + model, ClinicalEventColumn.value_as_number, sa.Float() + ), + _nullable_column( + model, + ClinicalEventColumn.value_as_concept_id, + sa.Integer(), + ), + _nullable_column( + model, ClinicalEventColumn.unit_concept_id, sa.Integer() + ), + ) + ) + return sa.select(*columns) + + +def canonical_event_union( + *models: type[Any], + include_values: bool = True, +) -> sa.Select[Any] | sa.CompoundSelect[Any]: + """Combine supported event models into one canonical ``UNION ALL`` query.""" + projections = [ + canonical_event_projection(model, include_values=include_values) + for model in models + ] + return _select_or_union_all( + projections, + error_message="canonical_event_union requires at least one model", + ) diff --git a/omop_alchemy/toolkit/core/modifiers/__init__.py b/omop_alchemy/toolkit/core/modifiers/__init__.py new file mode 100644 index 0000000..4ce7446 --- /dev/null +++ b/omop_alchemy/toolkit/core/modifiers/__init__.py @@ -0,0 +1,83 @@ +"""Canonical modifier projections, target validation, and selection.""" + +from .contracts import ( + ModifierColumn, + ModifierIdentity, + ModifierRow, + ModifierSelectionPolicy, + ModifierSelectionSpec, + ModifierTargetDiagnostic, + ModifierTargetDiagnosticCode, + ModifierTargetDiagnosticColumn, + ModifierTargetIdentity, + ValuedModifierRow, +) +from .metadata import ( + MODIFIER_SOURCE_MODEL_SPECS_BY_TABLE, + MODIFIER_TARGET_SPECS_BY_TABLE, + CDM_MODIFIER_SOURCE_MODELS, + ModifierTargetModelSpec, + UnsupportedModifierSourceModelError, + UnsupportedModifierTargetError, + modifier_source_model_spec, + modifier_target_model_spec, +) +from .projections import ( + canonical_modifier_projection, + canonical_modifier_union, +) +from .selection import ( + MODIFIER_RANK, + InvalidModifierSourceError, + modifier_order_expressions, + modifier_row_number, + ranked_modifier_select, + selected_modifier_select, +) +from .targets import ( + RESOLVED_TARGET_EVENT_ID, + RESOLVED_TARGET_FIELD_CONCEPT_ID, + RESOLVED_TARGET_PERSON_ID, + RESOLVED_TARGET_SOURCE_TABLE, + InvalidModifierTargetSourceError, + ModifierTargetQueries, + canonical_modifier_target_projection, + modifier_target_queries, +) + +__all__ = [ + "MODIFIER_SOURCE_MODEL_SPECS_BY_TABLE", + "MODIFIER_RANK", + "MODIFIER_TARGET_SPECS_BY_TABLE", + "RESOLVED_TARGET_EVENT_ID", + "RESOLVED_TARGET_FIELD_CONCEPT_ID", + "RESOLVED_TARGET_PERSON_ID", + "RESOLVED_TARGET_SOURCE_TABLE", + "CDM_MODIFIER_SOURCE_MODELS", + "InvalidModifierSourceError", + "InvalidModifierTargetSourceError", + "ModifierColumn", + "ModifierIdentity", + "ModifierRow", + "ModifierSelectionPolicy", + "ModifierSelectionSpec", + "ModifierTargetDiagnostic", + "ModifierTargetDiagnosticCode", + "ModifierTargetDiagnosticColumn", + "ModifierTargetIdentity", + "ModifierTargetModelSpec", + "ModifierTargetQueries", + "UnsupportedModifierSourceModelError", + "UnsupportedModifierTargetError", + "ValuedModifierRow", + "canonical_modifier_projection", + "canonical_modifier_target_projection", + "canonical_modifier_union", + "modifier_order_expressions", + "modifier_row_number", + "modifier_source_model_spec", + "modifier_target_model_spec", + "modifier_target_queries", + "ranked_modifier_select", + "selected_modifier_select", +] diff --git a/omop_alchemy/toolkit/core/modifiers/contracts.py b/omop_alchemy/toolkit/core/modifiers/contracts.py new file mode 100644 index 0000000..af08536 --- /dev/null +++ b/omop_alchemy/toolkit/core/modifiers/contracts.py @@ -0,0 +1,197 @@ +"""Side-effect-free contracts for canonical OMOP modifier rows. + +OMOP represents a modifier as an ordinary Measurement or Observation carrying +a polymorphic link to another row. Both ends of that relationship have scoped +identities: + +* a modifier ID is unique only within its source table; and +* a target event ID is meaningful only alongside the OMOP Field concept naming + the target table's primary-key field. + +The contracts below preserve those scopes after heterogeneous source tables are +projected into one query. They intentionally contain no ORM model registry; +physical model metadata lives in :mod:`.metadata`, while this module remains a +small vocabulary for rows, identities, selection, and diagnostics. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import date, datetime +from enum import StrEnum +from typing import Protocol, TypedDict, runtime_checkable + + +class ModifierColumn(StrEnum): + """Stable labels emitted by Measurement/Observation modifier projections. + + Native columns such as ``measurement_event_id`` and + ``observation_event_id`` converge on these names so downstream query code + never needs to branch on the physical modifier source. + """ + + person_id = "person_id" + modifier_id = "modifier_id" + modifier_date = "modifier_date" + modifier_datetime = "modifier_datetime" + modifier_concept_id = "modifier_concept_id" + modifier_source_table = "modifier_source_table" + target_event_id = "target_event_id" + target_field_concept_id = "target_field_concept_id" + value_as_number = "value_as_number" + value_as_concept_id = "value_as_concept_id" + unit_concept_id = "unit_concept_id" + value_as_string = "value_as_string" + + @classmethod + def required_columns(cls) -> tuple[ModifierColumn, ...]: + """Required projection labels in row-contract order.""" + return tuple(cls[name] for name in ModifierRow.__annotations__) + + @classmethod + def value_columns(cls) -> tuple[ModifierColumn, ...]: + """Nullable value labels, excluding inherited required fields.""" + return tuple(cls[name] for name in ValuedModifierRow.__annotations__) + + +@runtime_checkable +class ModifierRow(Protocol): + """Structural typing contract for the canonical identity and clinical row.""" + + person_id: int + modifier_id: int + modifier_date: date + modifier_datetime: datetime | None + modifier_concept_id: int + modifier_source_table: str + target_event_id: int | None + target_field_concept_id: int | None + + +@runtime_checkable +class ValuedModifierRow(ModifierRow, Protocol): + """Canonical modifier row extended with all nullable value representations.""" + + value_as_number: float | None + value_as_concept_id: int | None + unit_concept_id: int | None + value_as_string: str | None + + +@dataclass(frozen=True, order=True, slots=True) +class ModifierIdentity: + """Source-table-scoped identity of the modifier row itself.""" + + modifier_source_table: str + modifier_id: int + + def __post_init__(self) -> None: + if not self.modifier_source_table.strip(): + raise ValueError("modifier_source_table must not be empty") + + +@dataclass(frozen=True, order=True, slots=True) +class ModifierTargetIdentity: + """Field-concept-scoped identity of the row being modified.""" + + target_field_concept_id: int + target_event_id: int + + +class ModifierSelectionPolicy(StrEnum): + """Supported temporal directions after any caller-supplied priority.""" + + earliest = "earliest" + latest = "latest" + + +@dataclass(frozen=True, slots=True) +class ModifierSelectionSpec: + """Deterministic selection policy within a modifier target partition.""" + + policy: ModifierSelectionPolicy = ModifierSelectionPolicy.earliest + partition_by: tuple[str, ...] = ( + str(ModifierColumn.person_id), + str(ModifierColumn.target_field_concept_id), + str(ModifierColumn.target_event_id), + ) + date_column: str = str(ModifierColumn.modifier_date) + datetime_column: str = str(ModifierColumn.modifier_datetime) + stable_identity_columns: tuple[str, ...] = ( + str(ModifierColumn.modifier_source_table), + str(ModifierColumn.modifier_id), + ) + + def __post_init__(self) -> None: + names = ( + *self.partition_by, + self.date_column, + self.datetime_column, + *self.stable_identity_columns, + ) + if not self.partition_by or not self.stable_identity_columns: + raise ValueError("partition and stable identity columns must not be empty") + if any(not name.strip() for name in names): + raise ValueError("selection column names must not be empty") + if len(set(self.partition_by)) != len(self.partition_by): + raise ValueError("partition_by columns must be unique") + + +class ModifierTargetDiagnosticCode(StrEnum): + """Reasons a canonical modifier could not resolve to its supplied target.""" + + missing_target_identity = "missing_target_identity" + unsupported_target_field = "unsupported_target_field" + missing_target_event = "missing_target_event" + person_mismatch = "person_mismatch" + + +class ModifierTargetDiagnosticColumn(StrEnum): + """Stable output labels for advisory target-resolution diagnostics.""" + + diagnostic_code = "diagnostic_code" + modifier_source_table = "modifier_source_table" + modifier_id = "modifier_id" + target_field_concept_id = "target_field_concept_id" + target_event_id = "target_event_id" + message = "message" + + +class _ModifierTargetDiagnosticMapping(TypedDict): + diagnostic_code: str + modifier_source_table: str + modifier_id: int + target_field_concept_id: int | None + target_event_id: int | None + message: str + + +@dataclass(frozen=True, slots=True) +class ModifierTargetDiagnostic: + """Typed value representation of one target-resolution diagnostic row.""" + + diagnostic_code: ModifierTargetDiagnosticCode + modifier_source_table: str + modifier_id: int + target_field_concept_id: int | None + target_event_id: int | None + message: str + + @classmethod + def from_mapping( + cls, row: _ModifierTargetDiagnosticMapping + ) -> ModifierTargetDiagnostic: + return cls( + diagnostic_code=ModifierTargetDiagnosticCode( + row[ModifierTargetDiagnosticColumn.diagnostic_code.value] + ), + modifier_source_table=str( + row[ModifierTargetDiagnosticColumn.modifier_source_table.value] + ), + modifier_id=int(row[ModifierTargetDiagnosticColumn.modifier_id.value]), + target_field_concept_id=row[ + ModifierTargetDiagnosticColumn.target_field_concept_id.value + ], + target_event_id=row[ModifierTargetDiagnosticColumn.target_event_id.value], + message=str(row[ModifierTargetDiagnosticColumn.message.value]), + ) diff --git a/omop_alchemy/toolkit/core/modifiers/metadata.py b/omop_alchemy/toolkit/core/modifiers/metadata.py new file mode 100644 index 0000000..f1f174a --- /dev/null +++ b/omop_alchemy/toolkit/core/modifiers/metadata.py @@ -0,0 +1,151 @@ +"""Explicit model metadata for canonical OMOP modifier queries. + +The row contracts in :mod:`.contracts` describe data after projection and must +remain independent of mapped ORM models. This module owns the separate question +of which models may produce or receive those rows. + +``ModifierSourceMixin`` standardises the target link as the ``modifier_of_event_id`` +and ``modifier_of_field_concept_id`` hybrids +""" + +from __future__ import annotations + +from dataclasses import dataclass +from types import MappingProxyType +from typing import Any, Mapping + +from omop_alchemy.cdm.base import ModifierSourceMixin +from omop_alchemy.cdm.model.clinical import Measurement, Observation +from omop_alchemy.cdm.base.event_metadata import ( + ClinicalEventModelSpec, + UnsupportedClinicalEventModelError, +) +from omop_alchemy.cdm.base.errors import UnsupportedModelError +from omop_alchemy.cdm.model.clinical.event_metadata import ( + MODIFIER_TARGETS_BY_TABLE, + clinical_event_model_spec, +) + + +class UnsupportedModifierSourceModelError(UnsupportedModelError): + """Raised when a model cannot provide a canonical modifier projection.""" + + model_kind = "modifier model" + + +class UnsupportedModifierTargetError(UnsupportedModelError): + """Raised when a model cannot be a canonical modifier target.""" + + model_kind = "modifier target" + + +@dataclass(frozen=True, slots=True) +class ModifierTargetModelSpec: + """Native target identity and its canonical OMOP Field discriminator.""" + + event_id_attribute: str + event_field_concept_id: int + event_source_table: str + + +def _source_spec(model: type[Any]) -> ClinicalEventModelSpec: + """Resolve a source's own clinical-event metadata. + + A modifier source needs exactly the identity, date, datetime, concept and + table that a clinical event already describes, so the event spec is used + as-is. The modifier-flavoured renaming happens once, on the projection's + output labels, rather than in a parallel dataclass here. + """ + try: + event = clinical_event_model_spec(model) + except UnsupportedClinicalEventModelError as error: + raise UnsupportedModifierSourceModelError(model, error.reason) from error + if event.event_datetime_column is None: + raise UnsupportedModifierSourceModelError( + model, "must expose an event datetime column" + ) + return event + + +CDM_MODIFIER_SOURCE_MODELS: tuple[type[Any], ...] = (Measurement, Observation) + +MODIFIER_SOURCE_MODEL_SPECS_BY_TABLE: Mapping[str, ClinicalEventModelSpec] = ( + MappingProxyType( + { + model.__tablename__: _source_spec(model) + for model in CDM_MODIFIER_SOURCE_MODELS + } + ) +) + + +def _target_spec(target: type[Any]) -> ModifierTargetModelSpec: + # Every field is read off the target's ModifierTargetMixin declaration, so + # no table's identity metadata is restated here. + return ModifierTargetModelSpec( + event_id_attribute=target.__event_id_col__, + event_field_concept_id=target.modifier_field_concept_id(), + event_source_table=target.modifier_target_table(), + ) + + +MODIFIER_TARGET_SPECS_BY_TABLE: Mapping[str, ModifierTargetModelSpec] = ( + MappingProxyType( + { + table_name: _target_spec(target) + for table_name, target in MODIFIER_TARGETS_BY_TABLE.items() + } + ) +) + + +def modifier_source_model_spec(model: type[Any]) -> ClinicalEventModelSpec: + """Resolve and validate metadata for any model declaring the modifier link.""" + if not isinstance(model, type) or not hasattr(model, "__table__"): + raise UnsupportedModifierSourceModelError( + model, "expected a mapped ORM model class" + ) + if not issubclass(model, ModifierSourceMixin): + raise UnsupportedModifierSourceModelError( + model, "must declare the OMOP modifier link via ModifierSourceMixin" + ) + for declaration in ( + "__modifier_event_id_col__", + "__modifier_field_concept_id_col__", + ): + column_name = getattr(model, declaration, None) + if not isinstance(column_name, str) or not column_name.strip(): + raise UnsupportedModifierSourceModelError( + model, f"must declare {declaration}" + ) + if not hasattr(model, column_name): + raise UnsupportedModifierSourceModelError( + model, f"{declaration} names a missing column: {column_name}" + ) + # Only the bare built-ins share cached metadata. Views and subclasses may + # override event declarations and must resolve and validate their own spec. + if model in CDM_MODIFIER_SOURCE_MODELS: + return MODIFIER_SOURCE_MODEL_SPECS_BY_TABLE[model.__tablename__] + return _source_spec(model) + + +def modifier_target_model_spec(model: type[Any]) -> ModifierTargetModelSpec: + """Resolve and validate immutable metadata for a modifier target model.""" + if not isinstance(model, type) or not hasattr(model, "__table__"): + raise UnsupportedModifierTargetError(model, "expected a mapped ORM model class") + spec = MODIFIER_TARGET_SPECS_BY_TABLE.get(str(getattr(model, "__tablename__", ""))) + if spec is None: + raise UnsupportedModifierTargetError( + model, "is not a supported modifier target" + ) + missing = tuple( + name + for name in ("person_id", spec.event_id_attribute) + if not hasattr(model, name) + ) + if missing: + raise UnsupportedModifierTargetError( + model, + f"is missing required attributes: {', '.join(missing)}", + ) + return spec diff --git a/omop_alchemy/toolkit/core/modifiers/projections.py b/omop_alchemy/toolkit/core/modifiers/projections.py new file mode 100644 index 0000000..48d9d36 --- /dev/null +++ b/omop_alchemy/toolkit/core/modifiers/projections.py @@ -0,0 +1,74 @@ +"""Canonical SQLAlchemy projections for OMOP modifier-bearing tables.""" + +from __future__ import annotations + +from typing import Any, Mapping + +import sqlalchemy as sa + +from omop_alchemy.toolkit._utils import _nullable_column, _select_or_union_all + +from .contracts import ModifierColumn +from .metadata import ( + UnsupportedModifierSourceModelError, + modifier_source_model_spec, +) + + +# The SQL type each value position is cast to when a source has no such column. +# Keyed by column so the emission order below comes from the contract rather +# than from a second hand-maintained list. +_VALUE_COLUMN_TYPES: Mapping[ModifierColumn, sa.types.TypeEngine[Any]] = { + ModifierColumn.value_as_number: sa.Float(), + ModifierColumn.value_as_concept_id: sa.Integer(), + ModifierColumn.unit_concept_id: sa.Integer(), + ModifierColumn.value_as_string: sa.String(), +} + + +def canonical_modifier_projection( + model: type[Any], *, include_values: bool = True +) -> sa.Select[Any]: + """Project Measurement or Observation into one modifier row shape.""" + spec = modifier_source_model_spec(model) + datetime_column = spec.event_datetime_column + if datetime_column is None: # pragma: no cover - the spec rejects such a model + raise UnsupportedModifierSourceModelError( + model, "must expose an event datetime column" + ) + columns: list[sa.ColumnElement[Any]] = [ + model.person_id.label(str(ModifierColumn.person_id)), + getattr(model, spec.event_id_column).label(str(ModifierColumn.modifier_id)), + getattr(model, spec.event_date_column).label(str(ModifierColumn.modifier_date)), + getattr(model, datetime_column).label(str(ModifierColumn.modifier_datetime)), + getattr(model, spec.event_concept_id_column).label( + str(ModifierColumn.modifier_concept_id) + ), + sa.literal(spec.event_source_table).label( + str(ModifierColumn.modifier_source_table) + ), + # Guaranteed by ModifierSourceMixin, whatever the physical column name. + model.modifier_of_event_id.label(str(ModifierColumn.target_event_id)), + model.modifier_of_field_concept_id.label( + str(ModifierColumn.target_field_concept_id) + ), + ] + if include_values: + columns.extend( + _nullable_column(model, column, _VALUE_COLUMN_TYPES[column]) + for column in ModifierColumn.value_columns() + ) + return sa.select(*columns) + + +def canonical_modifier_union( + *models: type[Any], include_values: bool = True +) -> sa.Select[Any] | sa.CompoundSelect[Any]: + projections = [ + canonical_modifier_projection(model, include_values=include_values) + for model in models + ] + return _select_or_union_all( + projections, + error_message="canonical_modifier_union requires at least one model", + ) diff --git a/omop_alchemy/toolkit/core/modifiers/selection.py b/omop_alchemy/toolkit/core/modifiers/selection.py new file mode 100644 index 0000000..9031a94 --- /dev/null +++ b/omop_alchemy/toolkit/core/modifiers/selection.py @@ -0,0 +1,145 @@ +"""Deterministic selection of one canonical modifier per target partition.""" + +from __future__ import annotations + +from collections.abc import Sequence +from typing import Any + +import sqlalchemy as sa +from sqlalchemy.sql.selectable import FromClause, SelectBase + +from omop_alchemy.toolkit._utils import _as_from_clause, _require_columns +from omop_alchemy.toolkit.core._ranking import deterministic_row_number + +from .contracts import ModifierColumn, ModifierSelectionPolicy, ModifierSelectionSpec + +MODIFIER_RANK = "modifier_rank" + + +class InvalidModifierSourceError(ValueError): + """Raised when a modifier selection input lacks a required column.""" + + +def modifier_order_expressions( + columns: sa.sql.base.ReadOnlyColumnCollection[str, Any], + spec: ModifierSelectionSpec, + *, + priority: Sequence[sa.ColumnElement[Any]] = (), +) -> tuple[sa.ColumnElement[Any], ...]: + """Return the complete, portable order for a modifier selection policy.""" + required = { + *spec.partition_by, + spec.date_column, + spec.datetime_column, + *spec.stable_identity_columns, + } + _require_columns( + columns.keys(), + required, + role="modifier source", + error_type=InvalidModifierSourceError, + ) + direction = sa.desc if spec.policy is ModifierSelectionPolicy.latest else sa.asc + date_column = columns[spec.date_column] + datetime_column = columns[spec.datetime_column] + return ( + *priority, + date_column.is_(None).asc(), + direction(date_column), + datetime_column.is_(None).asc(), + direction(datetime_column), + *(columns[name].asc() for name in spec.stable_identity_columns), + ) + + +def modifier_row_number( + columns: sa.sql.base.ReadOnlyColumnCollection[str, Any], + spec: ModifierSelectionSpec, + *, + priority: Sequence[sa.ColumnElement[Any]] = (), + label: str = MODIFIER_RANK, +) -> sa.ColumnElement[int]: + """Build the deterministic window rank for canonical modifier columns.""" + order_by = modifier_order_expressions(columns, spec, priority=priority) + return deterministic_row_number( + partition_by=(columns[name] for name in spec.partition_by), + order_by=order_by, + label=label, + ) + + +def ranked_modifier_select( + source: FromClause | SelectBase, + spec: ModifierSelectionSpec = ModifierSelectionSpec(), + *, + priority: Sequence[sa.ColumnElement[Any]] = (), + rank_label: str = MODIFIER_RANK, +) -> sa.Select[Any]: + """Rank bound modifiers, with caller priorities preceding temporal policy.""" + modifiers = _as_from_clause(source, name="modifier_selection_source") + required = { + *spec.partition_by, + spec.date_column, + spec.datetime_column, + *spec.stable_identity_columns, + str(ModifierColumn.target_event_id), + str(ModifierColumn.target_field_concept_id), + } + _require_columns( + modifiers.c.keys(), + required, + role="modifier source", + error_type=InvalidModifierSourceError, + ) + + rank = modifier_row_number( + modifiers.c, + spec, + priority=priority, + label=rank_label, + ) + return sa.select(*modifiers.c, rank).where( + modifiers.c[str(ModifierColumn.target_event_id)].is_not(None), + modifiers.c[str(ModifierColumn.target_field_concept_id)].is_not(None), + ) + + +def selected_modifier_select( + source: FromClause | SelectBase, + spec: ModifierSelectionSpec = ModifierSelectionSpec(), + *, + priority: Sequence[sa.ColumnElement[Any]] = (), +) -> sa.Select[Any]: + """Select the first deterministically ranked modifier in each partition. + + Parameters + ---------- + source: + A selectable containing canonical modifier columns and target identity + columns. + spec: + Temporal direction, partition columns and stable identity columns used + to define the selection contract. + priority: + Optional SQL expressions placed before the temporal policy, such as a + domain-specific stage preference. + + Returns + ------- + sqlalchemy.sql.Select + A selectable containing the source columns, with one row at rank one + for each target partition. Rows missing either target identity column + are excluded before selection. + + Notes + ----- + The final tie-breakers come from ``spec.stable_identity_columns``. This + keeps the result deterministic when dates, datetimes and caller priorities + are equal. + """ + ranked = ranked_modifier_select(source, spec=spec, priority=priority).subquery( + "ranked_modifiers" + ) + return sa.select( + *(column for column in ranked.c if column.key != MODIFIER_RANK) + ).where(ranked.c[MODIFIER_RANK] == 1) diff --git a/omop_alchemy/toolkit/core/modifiers/targets.py b/omop_alchemy/toolkit/core/modifiers/targets.py new file mode 100644 index 0000000..4cb9d62 --- /dev/null +++ b/omop_alchemy/toolkit/core/modifiers/targets.py @@ -0,0 +1,218 @@ +"""Validate canonical modifier links against supported OMOP target tables.""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any + +import sqlalchemy as sa +from sqlalchemy.sql.selectable import FromClause, SelectBase + +from omop_alchemy.toolkit._utils import _as_from_clause, _require_columns +from omop_alchemy.toolkit.core.events import ClinicalEventColumn + +from .contracts import ( + ModifierColumn, + ModifierTargetDiagnosticCode, + ModifierTargetDiagnosticColumn, +) +from .metadata import modifier_target_model_spec +from .projections import canonical_modifier_projection + +RESOLVED_TARGET_PERSON_ID = "resolved_target_person_id" +RESOLVED_TARGET_EVENT_ID = "resolved_target_event_id" +RESOLVED_TARGET_FIELD_CONCEPT_ID = "resolved_target_field_concept_id" +RESOLVED_TARGET_SOURCE_TABLE = "resolved_target_source_table" + + +class InvalidModifierTargetSourceError(ValueError): + """Raised when a supplied projection lacks canonical columns.""" + + +@dataclass(frozen=True, slots=True) +class ModifierTargetQueries: + matches: sa.Select[Any] + diagnostics: SelectBase | None = None + + +def canonical_modifier_target_projection(model: type[Any]) -> sa.Select[Any]: + """Project a supported clinical event or Episode to target identity columns.""" + spec = modifier_target_model_spec(model) + return sa.select( + model.person_id.label(str(ClinicalEventColumn.person_id)), + getattr(model, spec.event_id_attribute).label( + str(ClinicalEventColumn.event_id) + ), + sa.literal(spec.event_field_concept_id).label( + str(ClinicalEventColumn.event_field_concept_id) + ), + sa.literal(spec.event_source_table).label( + str(ClinicalEventColumn.event_source_table) + ), + ) + + +def modifier_target_queries( + modifier_source: type[Any] | FromClause | SelectBase, + target_source: type[Any] | FromClause | SelectBase, + *, + include_unmatched: bool = False, + diagnostics: bool = False, +) -> ModifierTargetQueries: + """Resolve valid target links and optionally explain rejected modifier rows. + + Parameters + ---------- + modifier_source: + A supported modifier model or a selectable exposing the canonical + modifier columns. + target_source: + A supported ORM target model or a selectable exposing target identity + columns. Selectables are treated as the caller's supplied scope. + include_unmatched: + If ``True``, retain modifier rows without a valid target in + ``matches``. Otherwise only valid links are returned. + diagnostics: + If ``True``, also build an advisory diagnostic selectable. Building + the queries does not execute either selectable. + + Returns + ------- + ModifierTargetQueries + ``matches`` contains the modifier columns plus resolved target + identity columns. ``diagnostics`` is ``None`` unless requested. + + Notes + ----- + A link is valid only when target event ID, target Field concept and person + ID all agree. For an ORM target model, diagnostics can report an + unsupported target field or a missing target event. For a caller-supplied + selectable, those absences may be caused by filtering, so only missing + identity and observed person mismatches are reported. + """ + modifiers = ( + canonical_modifier_projection(modifier_source).subquery("target_modifiers") + if isinstance(modifier_source, type) + else _as_from_clause(modifier_source, name="target_modifiers") + ) + target_spec = ( + modifier_target_model_spec(target_source) + if isinstance(target_source, type) + else None + ) + targets = ( + canonical_modifier_target_projection(target_source).subquery("modifier_targets") + if isinstance(target_source, type) + else _as_from_clause(target_source, name="modifier_targets") + ) + _require_columns( + modifiers.c.keys(), + tuple(str(column) for column in ModifierColumn.required_columns()), + role="modifier source", + error_type=InvalidModifierTargetSourceError, + ) + _require_columns( + targets.c.keys(), + tuple( + str(column) + for column in ( + ClinicalEventColumn.person_id, + ClinicalEventColumn.event_id, + ClinicalEventColumn.event_field_concept_id, + ClinicalEventColumn.event_source_table, + ) + ), + role="target source", + error_type=InvalidModifierTargetSourceError, + ) + + person = str(ModifierColumn.person_id) + event_id = str(ClinicalEventColumn.event_id) + event_field = str(ClinicalEventColumn.event_field_concept_id) + target_id = str(ModifierColumn.target_event_id) + target_field = str(ModifierColumn.target_field_concept_id) + valid_link = sa.and_( + modifiers.c[target_id] == targets.c[event_id], + modifiers.c[target_field] == targets.c[event_field], + modifiers.c[person] == targets.c[person], + ) + joined = modifiers.join(targets, valid_link, isouter=include_unmatched) + matches = sa.select( + *modifiers.c, + targets.c[person].label(RESOLVED_TARGET_PERSON_ID), + targets.c[event_id].label(RESOLVED_TARGET_EVENT_ID), + targets.c[event_field].label(RESOLVED_TARGET_FIELD_CONCEPT_ID), + targets.c[str(ClinicalEventColumn.event_source_table)].label( + RESOLVED_TARGET_SOURCE_TABLE + ), + ).select_from(joined) + + if not diagnostics: + return ModifierTargetQueries(matches=matches) + + has_identity = sa.and_( + modifiers.c[target_id].is_not(None), modifiers.c[target_field].is_not(None) + ) + any_event = sa.exists( + sa.select(1) + .select_from(targets) + .where( + targets.c[event_id] == modifiers.c[target_id], + targets.c[event_field] == modifiers.c[target_field], + ) + ) + same_person = sa.exists(sa.select(1).select_from(targets).where(valid_link)) + + def branch( + code: ModifierTargetDiagnosticCode, + condition: sa.ColumnElement[bool], + message: str, + ) -> sa.Select[Any]: + return sa.select( + sa.literal(str(code)).label( + str(ModifierTargetDiagnosticColumn.diagnostic_code) + ), + modifiers.c[str(ModifierColumn.modifier_source_table)], + modifiers.c[str(ModifierColumn.modifier_id)], + modifiers.c[target_field], + modifiers.c[target_id], + sa.literal(message).label(str(ModifierTargetDiagnosticColumn.message)), + ).where(condition) + + branches: list[sa.Select[Any]] = [ + branch( + ModifierTargetDiagnosticCode.missing_target_identity, + sa.not_(has_identity), + "modifier does not identify both a target row and target field", + ) + ] + if target_spec is not None: + supported = modifiers.c[target_field] == target_spec.event_field_concept_id + branches.append( + branch( + ModifierTargetDiagnosticCode.unsupported_target_field, + sa.and_(has_identity, sa.not_(supported)), + "target field concept is not supported by the supplied target", + ) + ) + # Absence from the target set proves the row does not exist only when + # the set is a whole target table. A caller-supplied selectable may be + # filtered, where a missing row is the filter's doing, not a defect. + branches.append( + branch( + ModifierTargetDiagnosticCode.missing_target_event, + sa.and_(has_identity, supported, sa.not_(any_event)), + "the identified target event does not exist", + ) + ) + else: + supported = sa.true() + # Safe for either source: the mismatch is observed on a row that is present. + branches.append( + branch( + ModifierTargetDiagnosticCode.person_mismatch, + sa.and_(has_identity, supported, any_event, sa.not_(same_person)), + "modifier and target event belong to different people", + ) + ) + return ModifierTargetQueries(matches=matches, diagnostics=sa.union_all(*branches)) diff --git a/omop_alchemy/toolkit/core/timeline/__init__.py b/omop_alchemy/toolkit/core/timeline/__init__.py index 7cb044e..d31d1c7 100644 --- a/omop_alchemy/toolkit/core/timeline/__init__.py +++ b/omop_alchemy/toolkit/core/timeline/__init__.py @@ -1,14 +1,15 @@ """Project a person's clinical records into one ordered event sequence. -Conditions, measurements, and drug exposures live in separate CDM tables +Conditions, measurements, drug exposures, and observations live in separate CDM tables with differently named date and value columns. Answering "what happened to this patient, in order" means reconciling them. ``Person_Timeline`` does that reconciliation and presents the result as a single list of events sorted by time. -Each event exposes a canonical time and value regardless of which table it -came from, so callers can iterate a patient's history without special- -casing per table:: +Each event implements the canonical row identity from ``toolkit.core.events`` +and adds timeline-specific interval, value, metadata, and serialisation +behaviour, so callers can iterate a patient's history without special-casing +per table:: from omop_alchemy.toolkit.core.timeline import Person_Timeline @@ -29,6 +30,7 @@ EventTime, EventValue, Measurement_Event, + Observation_Event, Person_Timeline, ) @@ -41,5 +43,6 @@ "EventTime", "EventValue", "Measurement_Event", + "Observation_Event", "Person_Timeline", ] diff --git a/omop_alchemy/toolkit/core/timeline/event_timeline.py b/omop_alchemy/toolkit/core/timeline/event_timeline.py index 16fce08..ca5abce 100644 --- a/omop_alchemy/toolkit/core/timeline/event_timeline.py +++ b/omop_alchemy/toolkit/core/timeline/event_timeline.py @@ -1,18 +1,28 @@ - -from omop_alchemy.cdm.model.clinical import Measurement, Person, Condition_Occurrence, Drug_Exposure +from omop_alchemy.cdm.model.clinical import ( + Measurement, + Person, + Condition_Occurrence, + Drug_Exposure, + Observation, +) from sqlalchemy.orm import object_session from sqlalchemy import select from datetime import datetime, time, date -from typing import Optional, Mapping, Any, List +from typing import Optional, Mapping, Any, List, cast import json from dataclasses import dataclass from typing import Protocol, Union, Literal +from types import EllipsisType + +from omop_alchemy.cdm.model.clinical.event_metadata import clinical_event_model_spec +from omop_alchemy.toolkit.core.events import ClinicalEventRow TemporalKind = Literal["point", "interval"] EventValueType = Literal["numeric", "concept", "string", "none"] + @dataclass(frozen=True) class EventValue: type: EventValueType @@ -32,23 +42,22 @@ class EventTime: """ Canonical temporal representation for an event. """ + start: datetime end: Optional[datetime] = None @property def kind(self) -> TemporalKind: return "interval" if self.end is not None else "point" - -class ClinicalEventProtocol(Protocol): + +class ClinicalEventProtocol(ClinicalEventRow, Protocol): """ Interface for ORM rows that can be projected into a patient timeline. """ - @property - def person_id(self) -> int: ... - """Primary clinical concept driving the event""" + @property def concept_id(self) -> int: ... @@ -56,19 +65,21 @@ def concept_id(self) -> int: ... Canonical event time. Handles date vs datetime, start-only vs start+end. """ + @property - def event_time(self) -> EventTime:... + def event_time(self) -> EventTime: ... """ Numeric or categorical values associated with the event. Used for feature construction (e.g. value_as_number). """ - def event_value(self) -> EventValue: ... + def event_value(self) -> EventValue: ... """ Non-feature metadata (units, source value, modifiers). """ + def event_metadata(self) -> Mapping[str, Any]: ... def to_dict(self) -> dict[str, Any]: ... @@ -77,6 +88,9 @@ def to_json(self) -> str: ... @dataclass class EventMapping: + event_id_field: str + event_field_concept_id: int + event_source_table: str concept_field: str start_date_field: str start_datetime_field: Optional[str] = None @@ -84,6 +98,43 @@ class EventMapping: end_datetime_field: Optional[str] = None value_fields: Optional[List[str]] = None + @classmethod + def from_model( + cls, + model: type[Any], + *, + end_date_field: str | None | EllipsisType = ..., + end_datetime_field: str | None | EllipsisType = ..., + value_fields: list[str] | None = None, + ) -> "EventMapping": + """Build event fields from shared CDM metadata, including intervals. + + Equal start/end column declarations denote a point event. Omitted + endpoint arguments infer independent end columns; a string overrides + the corresponding column, and explicit None disables that endpoint. + """ + spec = clinical_event_model_spec(model) + return cls( + event_id_field=spec.event_id_column, + event_field_concept_id=spec.event_field_concept_id, + event_source_table=spec.event_source_table, + concept_field=spec.event_concept_id_column, + start_date_field=spec.event_date_column, + start_datetime_field=spec.event_datetime_column, + end_date_field=( + spec.event_end_date_column + if isinstance(end_date_field, EllipsisType) + else end_date_field + ), + end_datetime_field=( + spec.event_end_datetime_column + if isinstance(end_datetime_field, EllipsisType) + else end_datetime_field + ), + value_fields=value_fields, + ) + + def _as_datetime(d: date | datetime | None, *, end: bool = False) -> datetime | None: if d is None: return None @@ -91,14 +142,46 @@ def _as_datetime(d: date | datetime | None, *, end: bool = False) -> datetime | return d return datetime.combine(d, time.max if end else time.min) + class ClinicalEvent: - _mapping: EventMapping + @property + def event_id(self) -> int: + """Primary key value within the mapped source table.""" + return getattr(self, self._mapping.event_id_field) + @property def concept_id(self) -> int: return getattr(self, self._mapping.concept_field) - + + @property + def event_concept_id(self) -> int: + """Canonical projection name for the timeline event's clinical concept.""" + return self.concept_id + + @property + def event_source_table(self) -> str: + """OMOP source table that scopes ``event_id``.""" + return self._mapping.event_source_table + + @property + def event_field_concept_id(self) -> int: + """OMOP Field concept identifying the source event ID column.""" + return self._mapping.event_field_concept_id + + @property + def event_date(self) -> date: + """Date component used by canonical event projections.""" + value = getattr(self, self._mapping.start_date_field) + return value.date() if isinstance(value, datetime) else value + + @property + def event_datetime(self) -> datetime | None: + """Source datetime when the event table carries one, otherwise ``None``.""" + field = self._mapping.start_datetime_field + return getattr(self, field) if field else None + def event_value(self) -> EventValue: fields = self._mapping.value_fields or [] @@ -108,17 +191,27 @@ def event_value(self) -> EventValue: if value is None: continue - if 'concept' in field.lower() and 'number' not in field.lower() and isinstance(value, int) and value != 0: + if ( + "concept" in field.lower() + and "number" not in field.lower() + and isinstance(value, int) + and value != 0 + ): return EventValue(type="concept", value=value) - - if 'number' in field.lower() and isinstance(value, (int, float)): + + # OMOP quantity fields are numeric even though their names do not + # contain ``number``. Keep the mapping semantic here so callers do + # not need table-specific special cases when serialising timelines. + if ( + "number" in field.lower() or field.lower() == "quantity" + ) and isinstance(value, (int, float)): return EventValue(type="numeric", value=value) - if 'string' in field.lower() and isinstance(value, str) and value.strip(): + if "string" in field.lower() and isinstance(value, str) and value.strip(): return EventValue(type="string", value=value) return EventValue(type="none", value=None) - + @property def event_time(self) -> EventTime: m = self._mapping @@ -128,7 +221,9 @@ def event_time(self) -> EventTime: if start is None: start = _as_datetime(getattr(self, m.start_date_field), end=False) if start is None: - raise ValueError(f"{self.__class__.__name__}: could not resolve event start time") + raise ValueError( + f"{self.__class__.__name__}: could not resolve event start time" + ) end = None if m.end_datetime_field: end = getattr(self, m.end_datetime_field) @@ -137,14 +232,17 @@ def event_time(self) -> EventTime: if end is not None and end <= start: end = None return EventTime(start=start, end=end) - + def event_metadata(self) -> Mapping[str, Any]: return {} - - def __repr__(self: ClinicalEventProtocol) -> str: # ty: ignore[invalid-method-override] - et = self.event_time - ev = self.event_value() + def __repr__(self) -> str: + # The ORM event subclasses provide the row fields described by the + # protocol; the base class keeps that relationship structural so it + # does not need to inherit from a mapped row class. + event = cast(ClinicalEventProtocol, self) + et = event.event_time + ev = event.event_value() # time string if et.end is None: @@ -163,75 +261,61 @@ def __repr__(self: ClinicalEventProtocol) -> str: # ty: ignore[invalid-method-o value_str = "∅" return ( - f"<{self.__class__.__name__} " - f"person={self.person_id} " - f"concept={self.concept_id} " + f"<{event.__class__.__name__} " + f"person={event.person_id} " + f"concept={event.concept_id} " f"time={time_str} " f"value={value_str}>" ) - - def to_dict(self: ClinicalEventProtocol) -> dict[str, Any]: - et = self.event_time - ev = self.event_value() + + def to_dict(self) -> dict[str, Any]: + event = cast(ClinicalEventProtocol, self) + et = event.event_time + ev = event.event_value() return { - "person_id": self.person_id, - "concept_id": self.concept_id, + "person_id": event.person_id, + "event_id": event.event_id, + "event_source_table": event.event_source_table, + "event_field_concept_id": event.event_field_concept_id, + "event_concept_id": event.concept_id, "event_start": et.start.isoformat(), "event_end": et.end.isoformat() if et.end else None, "value": { "type": ev.type, "value": ev.value, }, - "metadata": dict(self.event_metadata() or {}), + "metadata": dict(event.event_metadata() or {}), } - def to_json(self: ClinicalEventProtocol) -> str: + def to_json(self) -> str: return json.dumps(self.to_dict(), ensure_ascii=False) - -class Condition_Event(Condition_Occurrence, ClinicalEvent): - _mapping = EventMapping( - concept_field="condition_concept_id", - start_date_field="condition_start_date", - start_datetime_field="condition_start_datetime", - end_date_field="condition_end_date", - end_datetime_field="condition_end_datetime", - ) +class Condition_Event(ClinicalEvent, Condition_Occurrence): + _mapping = EventMapping.from_model(Condition_Occurrence) class Measurement_Event(ClinicalEvent, Measurement): - - _mapping = EventMapping( - concept_field="measurement_concept_id", - start_date_field="measurement_date", - start_datetime_field="measurement_datetime", + _mapping = EventMapping.from_model( + Measurement, value_fields=[ "value_as_concept_id", "value_as_number", - "value_as_string", ], ) def event_metadata(self) -> dict[str, Optional[int]]: - metadata = { - "unit_concept_id": self.unit_concept_id - } + metadata = {"unit_concept_id": self.unit_concept_id} return metadata -class Drug_Exposure_Event(Drug_Exposure, ClinicalEvent): - - _mapping = EventMapping( - concept_field="drug_concept_id", - start_date_field="drug_exposure_start_date", - start_datetime_field="drug_exposure_start_datetime", - end_date_field="drug_exposure_end_date", - end_datetime_field="drug_exposure_end_datetime", +class Drug_Exposure_Event(ClinicalEvent, Drug_Exposure): + _mapping = EventMapping.from_model( + Drug_Exposure, value_fields=["quantity"], ) - + def event_metadata(self) -> Mapping[str, Any]: metadata = { "route_source_value": self.route_source_value, @@ -240,10 +324,20 @@ def event_metadata(self) -> Mapping[str, Any]: return metadata -class Person_Timeline(Person): +class Observation_Event(ClinicalEvent, Observation): + _mapping = EventMapping.from_model( + Observation, + value_fields=["value_as_concept_id", "value_as_number", "value_as_string"], + ) - EVENT_TABLES = (Measurement_Event, Condition_Event, Drug_Exposure_Event) +class Person_Timeline(Person): + EVENT_TABLES = ( + Measurement_Event, + Condition_Event, + Drug_Exposure_Event, + Observation_Event, + ) @property def events(self) -> list[ClinicalEvent]: @@ -254,20 +348,17 @@ def events(self) -> list[ClinicalEvent]: events: list[ClinicalEvent] = [] for EventCls in self.EVENT_TABLES: - stmt = ( - select(EventCls) - .where(EventCls.person_id == self.person_id) - ) + stmt = select(EventCls).where(EventCls.person_id == self.person_id) events.extend(session.execute(stmt).scalars()) return events - + @property def timeline(self) -> list[ClinicalEvent]: return sorted( self.events, key=lambda e: e.event_time.start, ) - + def to_json(self) -> list[str]: # ty: ignore[invalid-method-override] - return [e.to_json() for e in self.timeline] # ty: ignore[invalid-argument-type] \ No newline at end of file + return [e.to_json() for e in self.timeline] diff --git a/omop_alchemy/toolkit/episodes/derivation/__init__.py b/omop_alchemy/toolkit/episodes/derivation/__init__.py index 8ae225c..e87eecf 100644 --- a/omop_alchemy/toolkit/episodes/derivation/__init__.py +++ b/omop_alchemy/toolkit/episodes/derivation/__init__.py @@ -1,15 +1,98 @@ """Construct episodes and resolve the relationships between them. Episodes in the CDM are rows that reference each other through -``episode_parent_id`` and reach clinical facts through ``Episode_Event``. -Turning that into something queryable — a regimen with its cycles, a -diagnosis with the treatment that followed — is this tier's job: queries -that select episodes by concept, join parent and child episodes into a -single result so a hierarchy can be read in one pass, and establish the -date windows relating one episode to another, written against the raw -``Episode``/``Episode_Event`` tables rather than any materialised view. - -Not yet populated in this package. The equivalent built against -materialised-view subclasses lives in ``omop-constructs``; nothing has -moved into ``toolkit`` yet. +``episode_parent_id`` and resolve clinical facts through ``Episode_Event``. + +The public contracts in this area define episode-attachment identities and +policies used by query builders. Shared clinical-event row names and identities +live in ``toolkit.core.events``. + +Projection, attachment, and ranking helpers return SQLAlchemy statements or +expressions without executing them. """ + +from .attachments import ( + ATTACHMENT_METHOD, + EpisodeAttachmentQueries, + InvalidAttachmentSourceError, + episode_attachment_queries, +) +from .contracts import ( + CANONICAL_ATTACHMENT_DIAGNOSTIC_COLUMNS, + CANONICAL_EPISODE_COLUMNS, + CANONICAL_EPISODE_OPTIONAL_COLUMNS, + AttachmentDiagnosticCode, + AttachmentDiagnosticColumn, + EpisodeColumn, + EpisodeAttachmentDiagnostic, + EpisodeAttachmentIdentity, + EpisodeAttachmentMethod, + EpisodeAttachmentPolicy, + EpisodeWindowSpec, + ObservationSelectionPolicy, + ObservationSelectionSpec, + TemporalRankingSpec, + TemporalSelectionPolicy, + TemporalSidePreference, +) +from .observations import ( + observation_eligibility_predicate, + observation_order_expressions, + observation_row_number, + ranked_observation_select, +) +from .structure import ( + canonical_episode_projection, + direct_episode_relationship_projection, + episode_descendants, + episode_event_hierarchy_projection, +) +from .temporal import ( + absolute_day_delta, + bounded_temporal_predicate, + episode_window_bounds, + episode_window_predicate, + shift_date, + signed_day_delta, + temporal_order_expressions, + temporal_row_number, +) + +__all__ = [ + "AttachmentDiagnosticCode", + "AttachmentDiagnosticColumn", + "CANONICAL_ATTACHMENT_DIAGNOSTIC_COLUMNS", + "CANONICAL_EPISODE_COLUMNS", + "CANONICAL_EPISODE_OPTIONAL_COLUMNS", + "EpisodeColumn", + "EpisodeAttachmentDiagnostic", + "EpisodeAttachmentIdentity", + "ATTACHMENT_METHOD", + "EpisodeAttachmentMethod", + "EpisodeAttachmentPolicy", + "EpisodeAttachmentQueries", + "EpisodeWindowSpec", + "InvalidAttachmentSourceError", + "ObservationSelectionPolicy", + "ObservationSelectionSpec", + "TemporalRankingSpec", + "TemporalSelectionPolicy", + "TemporalSidePreference", + "absolute_day_delta", + "bounded_temporal_predicate", + "canonical_episode_projection", + "direct_episode_relationship_projection", + "episode_attachment_queries", + "episode_descendants", + "episode_event_hierarchy_projection", + "episode_window_bounds", + "episode_window_predicate", + "observation_eligibility_predicate", + "observation_order_expressions", + "observation_row_number", + "ranked_observation_select", + "shift_date", + "signed_day_delta", + "temporal_order_expressions", + "temporal_row_number", +] diff --git a/omop_alchemy/toolkit/episodes/derivation/attachments.py b/omop_alchemy/toolkit/episodes/derivation/attachments.py new file mode 100644 index 0000000..56dec93 --- /dev/null +++ b/omop_alchemy/toolkit/episodes/derivation/attachments.py @@ -0,0 +1,522 @@ +"""Explicit-first event-to-episode attachment queries.""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any, cast + +import sqlalchemy as sa +from sqlalchemy.sql.selectable import FromClause, SelectBase + +from omop_alchemy.cdm.model.structural import Episode, Episode_Event +from omop_alchemy.toolkit._utils import _as_from_clause, _require_columns +from omop_alchemy.toolkit.core.events import ( + ClinicalEventColumn, + canonical_event_projection, +) +from .contracts import ( + CANONICAL_ATTACHMENT_DIAGNOSTIC_COLUMNS, + AttachmentDiagnosticCode, + AttachmentDiagnosticColumn, + EpisodeAttachmentMethod, + EpisodeAttachmentPolicy, + EpisodeColumn, + EpisodeWindowSpec, + TemporalRankingSpec, +) +from .structure import canonical_episode_projection +from .temporal import episode_window_predicate, temporal_row_number + + +ATTACHMENT_EPISODE_ID = "episode_id" +ATTACHMENT_METHOD = "attachment_method" +_FALLBACK_RANK = "_fallback_rank" +_FALLBACK_CANDIDATE_COUNT = "_fallback_candidate_count" +_IDENTITY_RANK = "_attachment_identity_rank" + + +class InvalidAttachmentSourceError(ValueError): + """Raised when an attachment input does not expose its required columns.""" + + +@dataclass(frozen=True, slots=True) +class EpisodeAttachmentQueries: + """Attachment results and, when requested, their advisory diagnostics. + + ``attachments`` preserves the input event columns and appends ``episode_id`` + and ``attachment_method``. Its executable uniqueness key is + ``(event_source_table, event_id, episode_id)``. + + ``diagnostics`` is ``None`` unless requested. Diagnostics explain rejected + explicit links and fallback outcomes; they do not alter attachment rows. + """ + + attachments: sa.Select[Any] + diagnostics: sa.Select[Any] | None = None + + +def _event_source(source: type[Any] | FromClause | SelectBase) -> FromClause: + if isinstance(source, type): + return canonical_event_projection(source).subquery("attachment_events") + return _as_from_clause(source, name="attachment_events") + + +def _episode_source( + source: type[Episode] | FromClause | SelectBase, +) -> FromClause: + if isinstance(source, type): + model = cast(type[Episode], source) + return canonical_episode_projection(model).subquery("attachment_episodes") + return _as_from_clause(source, name="attachment_episodes") + + +def _episode_event_source( + source: type[Episode_Event] | FromClause | SelectBase, +) -> FromClause: + if isinstance(source, type): + model = cast(type[Episode_Event], source) + return model.__table__ + return _as_from_clause(source, name="attachment_episode_events") + + +def _same_event(left: FromClause, right: FromClause) -> sa.ColumnElement[bool]: + return sa.and_( + left.c[str(ClinicalEventColumn.event_source_table)] + == right.c[str(ClinicalEventColumn.event_source_table)], + left.c[str(ClinicalEventColumn.event_id)] + == right.c[str(ClinicalEventColumn.event_id)], + ) + + +def _not_exists_for_event( + source: FromClause, + keys: FromClause, +) -> sa.ColumnElement[bool]: + return sa.not_( + sa.exists(sa.select(1).select_from(keys).where(_same_event(source, keys))) + ) + + +def _attachment_diagnostics( + events: FromClause, + episodes: FromClause, + episode_events: FromClause, + valid_explicit: FromClause, + valid_explicit_event_keys: FromClause, + fallback_candidates: FromClause | None, + *, + policy: EpisodeAttachmentPolicy, +) -> sa.Select[Any]: + event_id = str(ClinicalEventColumn.event_id) + source_table = str(ClinicalEventColumn.event_source_table) + event_field = str(ClinicalEventColumn.event_field_concept_id) + episode_id = str(EpisodeColumn.episode_id) + person_id = str(ClinicalEventColumn.person_id) + link_field = "episode_event_field_concept_id" + + def diagnostic_literals( + code: AttachmentDiagnosticCode, + *, + linked_field: sa.ColumnElement[Any], + linked_episode_id: sa.ColumnElement[Any], + candidate_count: sa.ColumnElement[Any], + message: str, + ) -> tuple[sa.ColumnElement[Any], ...]: + return ( + sa.literal(str(code)).label( + str(AttachmentDiagnosticColumn.diagnostic_code) + ), + events.c[source_table].label(source_table), + events.c[event_id].label(event_id), + events.c[event_field].label(event_field), + linked_field.label( + str(AttachmentDiagnosticColumn.linked_event_field_concept_id) + ), + linked_episode_id.label(episode_id), + candidate_count.label(str(AttachmentDiagnosticColumn.candidate_count)), + sa.literal(message).label(str(AttachmentDiagnosticColumn.message)), + ) + + null_integer = sa.cast(sa.null(), sa.Integer()) + # A discriminator-correct link is still rejected when it crosses people; + # this protects downstream episode grains from attaching another person's + # otherwise valid event row. + person_mismatches = ( + sa.select( + *diagnostic_literals( + AttachmentDiagnosticCode.person_mismatch, + linked_field=episode_events.c[link_field], + linked_episode_id=episode_events.c[episode_id], + candidate_count=null_integer, + message="explicit link connects an event and episode belonging to different people", + ) + ) + .select_from( + events.join( + episode_events, + sa.and_( + events.c[event_id] == episode_events.c[event_id], + events.c[event_field] == episode_events.c[link_field], + ), + ).join( + episodes, + episodes.c[episode_id] == episode_events.c[episode_id], + ) + ) + .where(events.c[person_id] != episodes.c[person_id]) + ) + + diagnostic_branches: list[sa.Select[Any]] = [person_mismatches] + + if fallback_candidates is not None: + # Candidate counts are calculated before ranked fallback reduces the + # result. A selected row can therefore still explain that another + # eligible episode existed and was resolved by policy. + candidate_keys = ( + sa.select( + fallback_candidates.c[source_table], + fallback_candidates.c[event_id], + ) + .distinct() + .cte("fallback_candidate_event_keys") + ) + ambiguous = ( + sa.select( + *diagnostic_literals( + AttachmentDiagnosticCode.ambiguous_fallback, + linked_field=null_integer, + linked_episode_id=null_integer, + candidate_count=fallback_candidates.c[_FALLBACK_CANDIDATE_COUNT], + message="more than one episode is eligible for fallback attachment", + ) + ) + .select_from( + events.join( + fallback_candidates, _same_event(events, fallback_candidates) + ) + ) + .where(fallback_candidates.c[_FALLBACK_CANDIDATE_COUNT] > 1) + .distinct() + ) + if not policy.permits_fallback_fanout: + diagnostic_branches.append(ambiguous) + else: + candidate_keys = None + + # The final diagnostic is event-relative: no valid explicit relationship + # and, where fallback is enabled, no episode admitted by the window. + no_candidate_conditions = [_not_exists_for_event(events, valid_explicit_event_keys)] + if candidate_keys is not None: + no_candidate_conditions.append(_not_exists_for_event(events, candidate_keys)) + no_candidate_message = ( + "event has no valid explicit link" + if policy is EpisodeAttachmentPolicy.explicit_only + else "event has no valid explicit link or fallback episode in the configured window" + ) + no_candidate = sa.select( + *diagnostic_literals( + AttachmentDiagnosticCode.no_candidate_episode, + linked_field=null_integer, + linked_episode_id=null_integer, + candidate_count=sa.literal(0), + message=no_candidate_message, + ) + ).where(*no_candidate_conditions) + diagnostic_branches.append(no_candidate) + + combined = sa.union_all(*diagnostic_branches).subquery("attachment_diagnostics") + return sa.select( + *(combined.c[str(column)] for column in CANONICAL_ATTACHMENT_DIAGNOSTIC_COLUMNS) + ).distinct() + + +def episode_attachment_queries( + events: type[Any] | FromClause | SelectBase, + *, + policy: EpisodeAttachmentPolicy, + episodes: type[Episode] | FromClause | SelectBase = Episode, + episode_events: type[Episode_Event] | FromClause | SelectBase = Episode_Event, + ranking: TemporalRankingSpec | None = None, + window: EpisodeWindowSpec = EpisodeWindowSpec(), + include_diagnostics: bool = False, +) -> EpisodeAttachmentQueries: + """Build explicit-first attachments from canonical event and episode inputs. + + Parameters + ---------- + events: + A supported event model or selectable exposing the canonical event + columns. + policy: + Whether to use explicit links only, ranked fallback, or every eligible + episode in the fallback window. + episodes: + An Episode model or selectable exposing episode identity, person and + date bounds. + episode_events: + An Episode_Event model or selectable containing episode/event links and + their Field-concept discriminator. + ranking: + Temporal ranking used by ``explicit_first_ranked``. It must be omitted + for policies that do not rank fallback candidates. + window: + Episode-relative date window used to admit fallback candidates. + include_diagnostics: + If ``True``, return an advisory diagnostic selectable as well as the + attachment query. Building the queries does not execute them. + + Returns + ------- + EpisodeAttachmentQueries + ``attachments`` preserves event columns and appends ``episode_id`` and + ``attachment_method``. ``diagnostics`` is ``None`` unless requested. + + Notes + ----- + Resolution proceeds in three stages: validate explicit links, admit + same-person fallback candidates within the episode-relative window, then + retain every admitted candidate or apply temporal ranking according to + ``policy``. Explicit links suppress fallback only after the event ID, + Field-concept discriminator, episode ID and person all agree; the final + result is deduplicated by event source, event ID and episode ID. Fallback + ambiguity counts distinct eligible episodes, even when inputs repeat rows. + + Diagnostics explain rejected links and fallback outcomes without changing + attachment rows. Because inputs may be filtered selectables, the builder + does not infer missing source rows or unsupported discriminators from their + absence. + """ + if policy.requires_fallback_ranking and ranking is None: + raise ValueError("explicit_first_ranked requires a temporal ranking") + if not policy.requires_fallback_ranking and ranking is not None: + raise ValueError(f"{policy} does not use a temporal ranking") + + event_source = _event_source(events) + episode_source = _episode_source(episodes) + link_source = _episode_event_source(episode_events) + event_names = tuple(column.key for column in event_source.c) + + _require_columns( + event_source.c.keys(), + tuple(str(column) for column in ClinicalEventColumn.required_columns()), + role="events", + error_type=InvalidAttachmentSourceError, + ) + _require_columns( + episode_source.c.keys(), + ( + str(EpisodeColumn.episode_id), + str(EpisodeColumn.person_id), + str(EpisodeColumn.episode_start_date), + str(EpisodeColumn.episode_end_date), + ), + role="episodes", + error_type=InvalidAttachmentSourceError, + ) + _require_columns( + link_source.c.keys(), + ("episode_id", "event_id", "episode_event_field_concept_id"), + role="episode_events", + error_type=InvalidAttachmentSourceError, + ) + for reserved in (ATTACHMENT_EPISODE_ID, ATTACHMENT_METHOD): + if reserved in event_names: + raise InvalidAttachmentSourceError( + f"events already contains reserved attachment column {reserved!r}" + ) + + event_id = str(ClinicalEventColumn.event_id) + event_date = str(ClinicalEventColumn.event_date) + event_field = str(ClinicalEventColumn.event_field_concept_id) + source_table = str(ClinicalEventColumn.event_source_table) + person_id = str(ClinicalEventColumn.person_id) + episode_id = str(EpisodeColumn.episode_id) + episode_person_id = str(EpisodeColumn.person_id) + episode_start = str(EpisodeColumn.episode_start_date) + episode_end = str(EpisodeColumn.episode_end_date) + + # stage 1 of episode resolution accepts an explicit link only when ID, Field + # discriminator, episode, and person agree. This is the sole point where an + # explicit link becomes authoritative enough to suppress date-based fallback. + valid_explicit = ( + sa.select( + *(event_source.c[name] for name in event_names), + episode_source.c[episode_id].label(ATTACHMENT_EPISODE_ID), + sa.literal(str(EpisodeAttachmentMethod.explicit)).label(ATTACHMENT_METHOD), + ) + .select_from( + event_source.join( + link_source, + sa.and_( + event_source.c[event_id] == link_source.c[event_id], + event_source.c[event_field] + == link_source.c["episode_event_field_concept_id"], + ), + ).join( + episode_source, + sa.and_( + episode_source.c[episode_id] == link_source.c[episode_id], + episode_source.c[episode_person_id] == event_source.c[person_id], + ), + ) + ) + .distinct() + .cte("valid_explicit_attachments") + ) + + # These keys distinguish events that already have authoritative links from + # those whose absence or fallback outcome still needs explanation. Build the + # relation once so both fallback suppression and diagnostics reference the + # same CTE rather than constructing duplicate DISTINCT projections. + valid_explicit_event_keys = ( + sa.select( + valid_explicit.c[source_table], + valid_explicit.c[event_id], + ) + .distinct() + .cte("valid_explicit_event_keys") + ) + + attachment_names = (*event_names, ATTACHMENT_EPISODE_ID, ATTACHMENT_METHOD) + explicit_select = sa.select(*(valid_explicit.c[name] for name in attachment_names)) + fallback_candidates: FromClause | None = None + attachment_branches: list[sa.Select[Any]] = [explicit_select] + + if policy.uses_fallback: + # episode resolution stage 2 records the complete table-scoped identity + # of every valid explicit event. The anti-existence check below must use + # both columns: event_id alone is never a cross-table identity in OMOP. + fallback_join = event_source.join( + episode_source, + sa.and_( + event_source.c[person_id] == episode_source.c[episode_person_id], + episode_window_predicate( + event_source.c[event_date], + episode_source.c[episode_start], + episode_source.c[episode_end], + window=window, + ), + ), + ) + # Count identities before ranking: repeated source or episode rows are + # not evidence that another distinct episode is eligible. + fallback_episode_keys = ( + sa.select( + event_source.c[source_table], + event_source.c[event_id], + episode_source.c[episode_id], + ) + .select_from(fallback_join) + .where(_not_exists_for_event(event_source, valid_explicit_event_keys)) + .distinct() + .cte("fallback_episode_keys") + ) + fallback_counts = ( + sa.select( + fallback_episode_keys.c[source_table], + fallback_episode_keys.c[event_id], + sa.func.count().label(_FALLBACK_CANDIDATE_COUNT), + ) + .group_by( + fallback_episode_keys.c[source_table], + fallback_episode_keys.c[event_id], + ) + .cte("fallback_episode_counts") + ) + fallback_columns: list[sa.ColumnElement[Any]] = [ + *(event_source.c[name] for name in event_names), + episode_source.c[episode_id].label(ATTACHMENT_EPISODE_ID), + sa.literal(str(EpisodeAttachmentMethod.fallback)).label(ATTACHMENT_METHOD), + fallback_counts.c[_FALLBACK_CANDIDATE_COUNT], + ] + if policy.requires_fallback_ranking: + assert ranking is not None # validated above + if ranking.stable_id_column not in episode_source.c: + raise InvalidAttachmentSourceError( + "episodes is missing temporal stable ID column: " + f"{ranking.stable_id_column}" + ) + # episode resolution stage 3 ranks only after window admission. A side + # preference is a deliberate clinical policy tier; the stable episode + # ID prevents tied dates from depending on database row order. + fallback_columns.append( + temporal_row_number( + episode_source.c[episode_start], + event_source.c[event_date], + episode_source.c[ranking.stable_id_column], + ranking, + partition_by=( + event_source.c[source_table], + event_source.c[event_id], + ), + label=_FALLBACK_RANK, + ) + ) + + # All fallback policies share the same finite episode-relative window. + # The policy decides whether every admitted episode survives or exactly + # one ranked candidate is retained. + fallback_candidates = ( + sa.select(*fallback_columns) + .select_from( + fallback_join.join( + fallback_counts, _same_event(event_source, fallback_counts) + ) + ) + .cte("fallback_attachment_candidates") + ) + selected_fallback = sa.select( + *(fallback_candidates.c[name] for name in attachment_names) + ) + if policy.requires_fallback_ranking: + selected_fallback = selected_fallback.where( + fallback_candidates.c[_FALLBACK_RANK] == 1 + ) + attachment_branches.append(selected_fallback) + + # The builder accepts arbitrary selectables as inputs, so their source keys + # are not necessarily database constraints. Enforce the documented output + # identity here and prefer explicit provenance if a custom input manages to + # present the same attachment through more than one branch. + combined = sa.union_all(*attachment_branches).cte("combined_episode_attachments") + ranked_attachments = sa.select( + *(combined.c[name] for name in attachment_names), + sa.func.row_number() + .over( + partition_by=( + combined.c[source_table], + combined.c[event_id], + combined.c[ATTACHMENT_EPISODE_ID], + ), + order_by=sa.case( + ( + combined.c[ATTACHMENT_METHOD] + == str(EpisodeAttachmentMethod.explicit), + 0, + ), + else_=1, + ), + ) + .label(_IDENTITY_RANK), + ).cte("deduplicated_episode_attachments") + attachments = sa.select( + *(ranked_attachments.c[name] for name in attachment_names) + ).where(ranked_attachments.c[_IDENTITY_RANK] == 1) + + diagnostics = None + if include_diagnostics: + # Diagnostics explain rejected links and fallback outcomes without + # changing attachment rows. Non-matching discriminators and missing + # source rows are intentionally out of scope: this builder may receive + # a filtered projection and cannot infer absence from an OMOP table. + # ResolvedEpisodeEvent owns complete target-table resolution. + diagnostics = _attachment_diagnostics( + event_source, + episode_source, + link_source, + valid_explicit, + valid_explicit_event_keys, + fallback_candidates, + policy=policy, + ) + return EpisodeAttachmentQueries(attachments=attachments, diagnostics=diagnostics) diff --git a/omop_alchemy/toolkit/episodes/derivation/contracts.py b/omop_alchemy/toolkit/episodes/derivation/contracts.py new file mode 100644 index 0000000..4397cea --- /dev/null +++ b/omop_alchemy/toolkit/episodes/derivation/contracts.py @@ -0,0 +1,296 @@ +"""Side-effect-free contracts for episode attachment and temporal SQL builders. + +The shared clinical-event row and identity contracts live in +``toolkit.core.events`` so timelines and episode builders can consume them +without reversing the toolkit import layers. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from enum import StrEnum +from typing import TypedDict + +from omop_alchemy.toolkit.core.events import ClinicalEventIdentity +from omop_alchemy.toolkit.episodes.handling.event_windowing import ( + DEFAULT_EPISODE_OPEN_END_FALLBACK_DAYS, + DEFAULT_EPISODE_WINDOW_DAYS_PRIOR, +) + + +class EpisodeColumn(StrEnum): + """Canonical labels emitted by an episode projection.""" + + episode_id = "episode_id" + person_id = "person_id" + episode_parent_id = "episode_parent_id" + episode_start_date = "episode_start_date" + episode_start_datetime = "episode_start_datetime" + episode_end_date = "episode_end_date" + episode_end_datetime = "episode_end_datetime" + episode_concept_id = "episode_concept_id" + episode_object_concept_id = "episode_object_concept_id" + episode_type_concept_id = "episode_type_concept_id" + episode_concept_name = "episode_concept_name" + + +CANONICAL_EPISODE_COLUMNS: tuple[EpisodeColumn, ...] = tuple( + column + for column in EpisodeColumn + if column is not EpisodeColumn.episode_concept_name +) +"""Columns exposed by the canonical episode projection.""" + +CANONICAL_EPISODE_OPTIONAL_COLUMNS: tuple[EpisodeColumn, ...] = ( + EpisodeColumn.episode_concept_name, +) +"""Opt-in descriptive columns exposed by an enriched episode projection.""" + + +@dataclass(frozen=True, order=True, slots=True) +class EpisodeAttachmentIdentity: + """Unique identity of one event attached to one episode.""" + + event_source_table: str + event_id: int + episode_id: int + + def __post_init__(self) -> None: + # Keep the directly constructed form as safe as ``from_event``: downstream + # result sets use this key for deduplication, so an empty source would erase + # the boundary that protects against cross-table ID collisions. + ClinicalEventIdentity(self.event_source_table, self.event_id) + + @classmethod + def from_event( + cls, + event: ClinicalEventIdentity, + *, + episode_id: int, + ) -> EpisodeAttachmentIdentity: + """Add an episode to an already canonical cross-table event identity.""" + return cls( + event_source_table=event.event_source_table, + event_id=event.event_id, + episode_id=episode_id, + ) + + @property + def event(self) -> ClinicalEventIdentity: + """The event portion of this attachment identity.""" + return ClinicalEventIdentity(self.event_source_table, self.event_id) + + +class EpisodeAttachmentPolicy(StrEnum): + """Named precedence and fallback-cardinality policies. + + A valid explicit link always wins in the two explicit-first policies. The + difference is what happens to an event that has no valid explicit link. + """ + + explicit_only = "explicit_only" + explicit_first_ranked = "explicit_first_ranked" + explicit_first_all_in_window = "explicit_first_all_in_window" + + @property + def uses_fallback(self) -> bool: + """Whether unlinked events may be attached by a date window.""" + return self is not EpisodeAttachmentPolicy.explicit_only + + @property + def permits_fallback_fanout(self) -> bool: + """Whether one fallback event may attach to several episodes.""" + return self is EpisodeAttachmentPolicy.explicit_first_all_in_window + + @property + def requires_fallback_ranking(self) -> bool: + """Whether fallback needs a separate temporal ranking specification.""" + return self is EpisodeAttachmentPolicy.explicit_first_ranked + + +class EpisodeAttachmentMethod(StrEnum): + """How an event-to-episode attachment was established.""" + + explicit = "explicit" + fallback = "fallback" + + +class AttachmentDiagnosticCode(StrEnum): + """Stable categories for explaining rejected or ambiguous attachment rows.""" + + person_mismatch = "person_mismatch" + no_candidate_episode = "no_candidate_episode" + ambiguous_fallback = "ambiguous_fallback" + + +class AttachmentDiagnosticColumn(StrEnum): + """Stable labels emitted by an attachment diagnostics query.""" + + diagnostic_code = "diagnostic_code" + event_source_table = "event_source_table" + event_id = "event_id" + event_field_concept_id = "event_field_concept_id" + linked_event_field_concept_id = "linked_event_field_concept_id" + episode_id = "episode_id" + candidate_count = "candidate_count" + message = "message" + + +CANONICAL_ATTACHMENT_DIAGNOSTIC_COLUMNS: tuple[AttachmentDiagnosticColumn, ...] = tuple( + AttachmentDiagnosticColumn +) +"""Columns exposed by an attachment diagnostics query.""" + + +class _EpisodeAttachmentDiagnosticMapping(TypedDict): + diagnostic_code: str + event_source_table: str + event_id: int + event_field_concept_id: int + linked_event_field_concept_id: int | None + episode_id: int | None + candidate_count: int | None + message: str + + +@dataclass(frozen=True, slots=True) +class EpisodeAttachmentDiagnostic: + """Typed advisory result returned by an attachment diagnostics query.""" + + code: AttachmentDiagnosticCode + event: ClinicalEventIdentity + event_field_concept_id: int + message: str + linked_event_field_concept_id: int | None = None + episode_id: int | None = None + candidate_count: int | None = None + + @classmethod + def from_mapping( + cls, row: _EpisodeAttachmentDiagnosticMapping + ) -> "EpisodeAttachmentDiagnostic": + """Convert one SQLAlchemy mapping result without leaking column-name handling.""" + return cls( + code=AttachmentDiagnosticCode( + row[AttachmentDiagnosticColumn.diagnostic_code.value] + ), + event=ClinicalEventIdentity( + event_source_table=row[ + AttachmentDiagnosticColumn.event_source_table.value + ], + event_id=row[AttachmentDiagnosticColumn.event_id.value], + ), + event_field_concept_id=row[ + AttachmentDiagnosticColumn.event_field_concept_id.value + ], + linked_event_field_concept_id=row[ + AttachmentDiagnosticColumn.linked_event_field_concept_id.value + ], + episode_id=row[AttachmentDiagnosticColumn.episode_id.value], + candidate_count=row[AttachmentDiagnosticColumn.candidate_count.value], + message=row[AttachmentDiagnosticColumn.message.value], + ) + + +@dataclass(frozen=True, slots=True) +class EpisodeWindowSpec: + """Finite episode-relative window used to admit fallback event candidates.""" + + days_prior: int = DEFAULT_EPISODE_WINDOW_DAYS_PRIOR + open_end_fallback_days: int = DEFAULT_EPISODE_OPEN_END_FALLBACK_DAYS + include_lower_bound: bool = True + include_upper_bound: bool = True + + def __post_init__(self) -> None: + if self.days_prior < 0: + raise ValueError("days_prior must be non-negative") + if self.open_end_fallback_days < 0: + raise ValueError("open_end_fallback_days must be non-negative") + + +class TemporalSelectionPolicy(StrEnum): + """How one row is selected from several temporal candidates.""" + + nearest = "nearest" + earliest = "earliest" + latest = "latest" + + +class TemporalSidePreference(StrEnum): + """Which side of an anchor is preferred before temporal ranking. + + Candidate dates are ranked relative to a caller-supplied anchor. For event + attachment where the event is the anchor and episode starts are candidates, + ``on_or_before_anchor`` prefers an episode that had already started. + """ + + none = "none" + on_or_before_anchor = "on_or_before_anchor" + on_or_after_anchor = "on_or_after_anchor" + + +@dataclass(frozen=True, slots=True) +class TemporalRankingSpec: + """Temporal ranking contract for SQL builders. + + A side preference, when present, is applied before the selection policy. + ``nearest`` then means the smallest absolute distance within that tier. + All policies use the named stable ID column as their final ascending + tie-breaker, so the same source rows cannot alternate across executions. + """ + + policy: TemporalSelectionPolicy + stable_id_column: str + side_preference: TemporalSidePreference = TemporalSidePreference.none + + def __post_init__(self) -> None: + if not self.stable_id_column.strip(): + raise ValueError("stable_id_column must not be empty") + + @property + def uses_absolute_distance(self) -> bool: + """Whether ranking uses absolute distance after any side-preference tier.""" + return self.policy is TemporalSelectionPolicy.nearest + + @property + def has_side_preference(self) -> bool: + """Whether candidates are tiered by their side of the anchor first.""" + return self.side_preference is not TemporalSidePreference.none + + +class ObservationSelectionPolicy(StrEnum): + """Supported deterministic choices for repeated longitudinal observations.""" + + earliest = "earliest" + latest = "latest" + latest_on_or_before_anchor = "latest_on_or_before_anchor" + + +@dataclass(frozen=True, slots=True) +class ObservationSelectionSpec: + """Partition and tie-break contract for repeated-observation selection. + + The default partition describes one observation concept for one person. + Callers may add an episode or another grouping field when their declared + result grain requires it. + """ + + policy: ObservationSelectionPolicy + partition_by: tuple[str, ...] = ("person_id", "observation_concept_id") + stable_id_column: str = "observation_id" + include_anchor_date: bool = True + + def __post_init__(self) -> None: + if not self.partition_by: + raise ValueError("partition_by must contain at least one column") + if any(not name.strip() for name in self.partition_by): + raise ValueError("partition_by column names must not be empty") + if len(set(self.partition_by)) != len(self.partition_by): + raise ValueError("partition_by column names must be unique") + if not self.stable_id_column.strip(): + raise ValueError("stable_id_column must not be empty") + + @property + def requires_anchor(self) -> bool: + """Whether selection requires a caller-supplied anchor date.""" + return self.policy is ObservationSelectionPolicy.latest_on_or_before_anchor diff --git a/omop_alchemy/toolkit/episodes/derivation/observations.py b/omop_alchemy/toolkit/episodes/derivation/observations.py new file mode 100644 index 0000000..787fa67 --- /dev/null +++ b/omop_alchemy/toolkit/episodes/derivation/observations.py @@ -0,0 +1,115 @@ +"""Reusable SQL selection for repeated longitudinal observations.""" + +from __future__ import annotations + +from typing import Any + +import sqlalchemy as sa + +from omop_alchemy.toolkit.core._ranking import deterministic_row_number +from .contracts import ObservationSelectionPolicy, ObservationSelectionSpec + + +def observation_eligibility_predicate( + observation_date: sa.ColumnElement[Any], + spec: ObservationSelectionSpec, + *, + anchor_date: sa.ColumnElement[Any] | None = None, +) -> sa.ColumnElement[bool]: + """Return the date predicate required by an observation selection policy.""" + if not spec.requires_anchor: + # Unanchored policies keep every source row eligible; callers can reuse + # the ranking builder for episode and person level observations + return sa.true() + if anchor_date is None: + # Missing anchor input is a configuration error, not an instruction to + # widen the selection and risk returning an unrelated observation. + raise ValueError(f"{spec.policy} requires anchor_date") + return ( + observation_date <= anchor_date + if spec.include_anchor_date + else observation_date < anchor_date + ) + + +def observation_order_expressions( + observation_date: sa.ColumnElement[Any], + stable_id: sa.ColumnElement[Any], + spec: ObservationSelectionSpec, +) -> tuple[sa.ColumnElement[Any], ...]: + """Build date and stable-ID ordering for repeated observations.""" + if spec.policy is ObservationSelectionPolicy.earliest: + date_order = observation_date.asc() + elif spec.policy in ( + ObservationSelectionPolicy.latest, + ObservationSelectionPolicy.latest_on_or_before_anchor, + ): + date_order = observation_date.desc() + else: # pragma: no cover - StrEnum construction prevents unknown policies + raise ValueError(f"Unsupported observation selection policy: {spec.policy}") + return date_order, stable_id.asc() + + +def observation_row_number( + columns: Any, + *, + observation_date_column: str, + spec: ObservationSelectionSpec, + label: str = "observation_rank", +) -> sa.ColumnElement[int]: + """Return a deterministic row number using the declared observation grain. + + The caller supplies column names rather than mapped attributes so this + helper works against raw OMOP sources and projected/aliased selects alike. + """ + try: + observation_date = columns[observation_date_column] + stable_id = columns[spec.stable_id_column] + partition_by = tuple(columns[name] for name in spec.partition_by) + except KeyError as error: + raise ValueError( + f"Observation selection column is missing: {error.args[0]}" + ) from error + return deterministic_row_number( + partition_by=partition_by, + order_by=observation_order_expressions(observation_date, stable_id, spec), + label=label, + ) + + +def ranked_observation_select( + source: sa.FromClause, + spec: ObservationSelectionSpec, + *, + observation_date_column: str = "observation_date", + anchor_date: sa.ColumnElement[Any] | None = None, + rank_label: str = "observation_rank", +) -> sa.Select[Any]: + """Select source columns with deterministic rank and eligibility. + + Eligibility is applied in the same select that computes the window rank, + keeping out-of-window rows from competing for rank one. The source's + existing columns are preserved and the rank is appended for downstream + projection or filtering. + """ + columns = source.c + try: + observation_date = columns[observation_date_column] + except KeyError as error: + raise ValueError( + f"Observation selection column is missing: {observation_date_column}" + ) from error + + rank = observation_row_number( + columns, + observation_date_column=observation_date_column, + spec=spec, + label=rank_label, + ) + return sa.select(*columns, rank).where( + observation_eligibility_predicate( + observation_date, + spec, + anchor_date=anchor_date, + ) + ) diff --git a/omop_alchemy/toolkit/episodes/derivation/structure.py b/omop_alchemy/toolkit/episodes/derivation/structure.py new file mode 100644 index 0000000..1ff2c69 --- /dev/null +++ b/omop_alchemy/toolkit/episodes/derivation/structure.py @@ -0,0 +1,187 @@ +"""Domain-neutral projections over episode hierarchies and linked events.""" + +from __future__ import annotations + +from typing import Any, cast + +import sqlalchemy as sa +import sqlalchemy.orm as so +from sqlalchemy.sql.selectable import FromClause, SelectBase + +from omop_alchemy.cdm.model.structural import Episode, Episode_Event +from omop_alchemy.cdm.model.vocabulary import Concept +from omop_alchemy.toolkit._utils import _as_from_clause + +from .contracts import CANONICAL_EPISODE_COLUMNS, EpisodeColumn + + +EpisodeSource = type[Episode] | FromClause | SelectBase +EpisodeEventSource = type[Episode_Event] | FromClause | SelectBase +EpisodeHierarchySource = type[Episode] | type[Episode_Event] | FromClause | SelectBase +# Hierarchy builders deliberately accept both mapped tables and pre-shaped +# selectables. This keeps filtering/aliasing at the caller boundary instead of +# forcing recursive queries to rediscover or override that source definition. + + +def _episode_source(source: EpisodeHierarchySource, *, name: str) -> FromClause: + # Mapped classes contribute only their table here; relationship-bearing ORM + # behavior is intentionally kept out of these SQL-only hierarchy builders. + if isinstance(source, type): + return cast(FromClause, getattr(source, "__table__")) + return _as_from_clause(source, name=name) + + +def canonical_episode_projection( + episode_model: EpisodeSource = Episode, + *, + include_concept_label: bool = False, +) -> sa.Select[Any]: + """Project stable episode fields, optionally including its concept name.""" + episodes = _episode_source(episode_model, name="episode_projection") + # Every output label comes from the shared contract so projected episodes + # can be consumed by attachments and hierarchy helpers without table-specific + # column knowledge. + columns = [ + *( + episodes.c[str(column)].label(str(column)) + for column in CANONICAL_EPISODE_COLUMNS + ) + ] + if not include_concept_label: + return sa.select(*columns).select_from(episodes) + + # Keep the episode row when vocabulary data is absent; labels are an + # optional presentation enrichment, not a filter on structural episodes. + episode_concept = so.aliased(Concept, name="episode_projection_concept") + columns.append( + episode_concept.concept_name.label(str(EpisodeColumn.episode_concept_name)) + ) + return ( + sa.select(*columns) + .select_from(episodes) + .join( + episode_concept, + episode_concept.concept_id + == episodes.c[str(EpisodeColumn.episode_concept_id)], + isouter=True, + ) + ) + + +def direct_episode_relationship_projection( + episode_model: EpisodeSource = Episode, +) -> sa.Select[Any]: + """Select direct parent-child pairs with a depth of one.""" + episodes = _episode_source(episode_model, name="episode_relationships") + # Episode IDs are joined within person so a malformed or imported dataset + # cannot connect two patients merely because their IDs collide. + parent = episodes.alias("parent_episode") + child = episodes.alias("child_episode") + return sa.select( + parent.c[str(EpisodeColumn.episode_id)].label("root_episode_id"), + child.c[str(EpisodeColumn.episode_id)].label(str(EpisodeColumn.episode_id)), + child.c[str(EpisodeColumn.episode_parent_id)].label( + str(EpisodeColumn.episode_parent_id) + ), + child.c[str(EpisodeColumn.person_id)].label(str(EpisodeColumn.person_id)), + sa.literal(1).label("depth"), + ).select_from( + parent.join( + child, + sa.and_( + child.c[str(EpisodeColumn.episode_parent_id)] + == parent.c[str(EpisodeColumn.episode_id)], + child.c[str(EpisodeColumn.person_id)] + == parent.c[str(EpisodeColumn.person_id)], + ), + ) + ) + + +def episode_descendants( + *, + root_episode_id: int | sa.ColumnElement[Any] | None = None, + episode_model: EpisodeSource = Episode, + include_root: bool = True, + max_depth: int = 100, + name: str = "episode_descendants", +) -> sa.CTE: + """Build a recursive root-to-descendant projection with bounded depth.""" + if max_depth < 0: + raise ValueError("max_depth must be non-negative") + + episodes = _episode_source(episode_model, name=f"{name}_source") + episode_id = str(EpisodeColumn.episode_id) + episode_parent_id = str(EpisodeColumn.episode_parent_id) + person_id = str(EpisodeColumn.person_id) + # The seed establishes the requested root and depth zero; the recursive + # branch walks only same-person parent links and is bounded for cyclic data. + seed = sa.select( + episodes.c[episode_id].label("root_episode_id"), + episodes.c[episode_id].label(episode_id), + episodes.c[episode_parent_id].label(episode_parent_id), + episodes.c[person_id].label(person_id), + sa.literal(0).label("depth"), + ) + if root_episode_id is not None: + seed = seed.where(episodes.c[episode_id] == root_episode_id) + + hierarchy = seed.cte(f"{name}_walk", recursive=True) + child = episodes.alias(f"{name}_child") + hierarchy = hierarchy.union_all( + sa.select( + hierarchy.c.root_episode_id, + child.c[episode_id], + child.c[episode_parent_id], + child.c[person_id], + (hierarchy.c.depth + 1).label("depth"), + ) + .select_from( + hierarchy.join( + child, + sa.and_( + child.c[episode_parent_id] == hierarchy.c[episode_id], + child.c[person_id] == hierarchy.c[person_id], + ), + ) + ) + .where(hierarchy.c.depth < max_depth) + ) + if include_root: + return hierarchy + # Preserve the same CTE shape while removing only the seed rows from the + # public result, so downstream joins can use one stable hierarchy contract. + return sa.select(*hierarchy.c).where(hierarchy.c.depth > 0).cte(name) + + +def episode_event_hierarchy_projection( + *, + root_episode_id: int | sa.ColumnElement[Any] | None = None, + episode_model: EpisodeSource = Episode, + episode_event_model: EpisodeEventSource = Episode_Event, + include_root: bool = True, + max_depth: int = 100, +) -> sa.Select[Any]: + """Select linked events with their root episode, owning episode, and depth.""" + hierarchy = episode_descendants( + root_episode_id=root_episode_id, + episode_model=episode_model, + include_root=include_root, + max_depth=max_depth, + name="episode_event_descendants", + ) + event = _episode_source( + episode_event_model, + name="episode_event_hierarchy_events", + ) + # Keep both the root and owning episode: child-linked events should remain + # attributable to their direct owner while still supporting root-level + # episode analyses. + return sa.select( + hierarchy.c.root_episode_id, + hierarchy.c.episode_id.label("linked_episode_id"), + hierarchy.c.person_id, + hierarchy.c.depth.label("episode_depth"), + event.c["event_id"], + event.c["episode_event_field_concept_id"].label("event_field_concept_id"), + ).select_from(hierarchy.join(event, event.c.episode_id == hierarchy.c.episode_id)) diff --git a/omop_alchemy/toolkit/episodes/derivation/temporal.py b/omop_alchemy/toolkit/episodes/derivation/temporal.py new file mode 100644 index 0000000..56e567c --- /dev/null +++ b/omop_alchemy/toolkit/episodes/derivation/temporal.py @@ -0,0 +1,227 @@ +"""Portable SQL expressions for bounded windows and deterministic date ranking.""" + +from __future__ import annotations + +from collections.abc import Iterable +from typing import Any + +import sqlalchemy as sa +from sqlalchemy.ext.compiler import compiles +from sqlalchemy.sql.compiler import SQLCompiler +from sqlalchemy.sql.functions import FunctionElement + +from omop_alchemy.toolkit.core._ranking import deterministic_row_number +from .contracts import ( + EpisodeWindowSpec, + TemporalRankingSpec, + TemporalSelectionPolicy, + TemporalSidePreference, +) + + +class _SignedDayDelta(FunctionElement[int]): + """Dialect-specific whole-calendar-day difference used by ranking rules.""" + + type = sa.Integer() + inherit_cache = True + + +@compiles(_SignedDayDelta, "postgresql") +def _compile_signed_day_delta( + element: _SignedDayDelta, + compiler: SQLCompiler, + **kwargs: Any, +) -> str: + # PostgreSQL dates subtract to an integer; casting first makes timestamp + # inputs obey the toolkit's calendar-day contract instead of elapsed-time + # semantics. + candidate, anchor = list(element.clauses) + return ( + f"(CAST({compiler.process(candidate, **kwargs)} AS DATE) - " + f"CAST({compiler.process(anchor, **kwargs)} AS DATE))" + ) + + +@compiles(_SignedDayDelta, "sqlite") +def _compile_sqlite_signed_day_delta( + element: _SignedDayDelta, + compiler: SQLCompiler, + **kwargs: Any, +) -> str: + # SQLite has no date subtraction operator. Truncate both operands before + # julianday arithmetic so this stays equivalent to PostgreSQL for dates. + candidate, anchor = list(element.clauses) + return ( + "CAST((julianday(date(" + f"{compiler.process(candidate, **kwargs)})) - julianday(date(" + f"{compiler.process(anchor, **kwargs)}))) AS INTEGER)" + ) + + +class _ShiftDate(FunctionElement[Any]): + """Dialect-specific date shift used to construct episode window bounds.""" + + type = sa.Date() + inherit_cache = True + + +@compiles(_ShiftDate, "postgresql") +def _compile_shift_date( + element: _ShiftDate, + compiler: SQLCompiler, + **kwargs: Any, +) -> str: + # Keep the date cast on the value and integer cast on the offset explicit; + # this avoids PostgreSQL choosing timestamp/interval semantics implicitly. + value, days = list(element.clauses) + return ( + f"(CAST({compiler.process(value, **kwargs)} AS DATE) + " + f"CAST({compiler.process(days, **kwargs)} AS INTEGER))" + ) + + +@compiles(_ShiftDate, "sqlite") +def _compile_sqlite_shift_date( + element: _ShiftDate, + compiler: SQLCompiler, + **kwargs: Any, +) -> str: + # SQLite's portable equivalent is its date modifier syntax. The printf + # form keeps the offset bound as a SQL expression rather than interpolating + # it into generated SQL. + value, days = list(element.clauses) + return ( + f"date({compiler.process(value, **kwargs)}, " + f"printf('%+d days', {compiler.process(days, **kwargs)}))" + ) + + +def signed_day_delta( + candidate_date: sa.ColumnElement[Any], + anchor_date: sa.ColumnElement[Any], +) -> sa.ColumnElement[int]: + """Return candidate minus anchor in whole calendar days.""" + return _SignedDayDelta(candidate_date, anchor_date) + + +def absolute_day_delta( + candidate_date: sa.ColumnElement[Any], + anchor_date: sa.ColumnElement[Any], +) -> sa.ColumnElement[int]: + """Return absolute calendar-day distance between candidate and anchor.""" + return sa.func.abs(signed_day_delta(candidate_date, anchor_date)) + + +def shift_date( + value: sa.ColumnElement[Any], + *, + days: int, +) -> sa.ColumnElement[Any]: + """Shift a date by a fixed number of days on PostgreSQL or SQLite.""" + return _ShiftDate(value, sa.literal(days)) + + +def bounded_temporal_predicate( + value: sa.ColumnElement[Any], + lower_bound: sa.ColumnElement[Any], + upper_bound: sa.ColumnElement[Any], + *, + include_lower_bound: bool = True, + include_upper_bound: bool = True, +) -> sa.ColumnElement[bool]: + """Test a value against independently open or closed temporal bounds.""" + lower = value >= lower_bound if include_lower_bound else value > lower_bound + upper = value <= upper_bound if include_upper_bound else value < upper_bound + return sa.and_(lower, upper) + + +def episode_window_bounds( + episode_start_date: sa.ColumnElement[Any], + episode_end_date: sa.ColumnElement[Any], + *, + window: EpisodeWindowSpec = EpisodeWindowSpec(), +) -> tuple[sa.ColumnElement[Any], sa.ColumnElement[Any]]: + """Build the bounded SQL interval used for date-admitted episode facts. + + Closed episodes use their recorded end date. Open episodes use the + configured fallback horizon so an absent end date cannot silently produce + an unbounded join. + """ + lower = shift_date(episode_start_date, days=-window.days_prior) + upper = sa.func.coalesce( + episode_end_date, + shift_date(episode_start_date, days=window.open_end_fallback_days), + ) + return lower, upper + + +def episode_window_predicate( + event_date: sa.ColumnElement[Any], + episode_start_date: sa.ColumnElement[Any], + episode_end_date: sa.ColumnElement[Any], + *, + window: EpisodeWindowSpec = EpisodeWindowSpec(), +) -> sa.ColumnElement[bool]: + """Test whether an event lies inside a bounded episode-relative window.""" + lower, upper = episode_window_bounds( + episode_start_date, + episode_end_date, + window=window, + ) + return bounded_temporal_predicate( + event_date, + lower, + upper, + include_lower_bound=window.include_lower_bound, + include_upper_bound=window.include_upper_bound, + ) + + +def temporal_order_expressions( + candidate_date: sa.ColumnElement[Any], + anchor_date: sa.ColumnElement[Any], + stable_id: sa.ColumnElement[Any], + ranking: TemporalRankingSpec, +) -> tuple[sa.ColumnElement[Any], ...]: + """Build deterministic ordering for a temporal ranking contract.""" + order: list[sa.ColumnElement[Any]] = [] + # Side preference is a priority tier, not an additional date filter: the + # ranking contract may still select the nearest row on the other side. + if ranking.side_preference is TemporalSidePreference.on_or_before_anchor: + order.append(sa.case((candidate_date <= anchor_date, 0), else_=1).asc()) + elif ranking.side_preference is TemporalSidePreference.on_or_after_anchor: + order.append(sa.case((candidate_date >= anchor_date, 0), else_=1).asc()) + + if ranking.policy is TemporalSelectionPolicy.nearest: + order.append(absolute_day_delta(candidate_date, anchor_date).asc()) + elif ranking.policy is TemporalSelectionPolicy.earliest: + order.append(candidate_date.asc()) + elif ranking.policy is TemporalSelectionPolicy.latest: + order.append(candidate_date.desc()) + else: # pragma: no cover - StrEnum construction prevents unknown policies + raise ValueError(f"Unsupported temporal selection policy: {ranking.policy}") + # A stable source ID makes equal-date/equal-distance candidates reproducible. + order.append(stable_id.asc()) + return tuple(order) + + +def temporal_row_number( + candidate_date: sa.ColumnElement[Any], + anchor_date: sa.ColumnElement[Any], + stable_id: sa.ColumnElement[Any], + ranking: TemporalRankingSpec, + *, + partition_by: Iterable[sa.ColumnElement[Any]] = (), + label: str = "temporal_rank", +) -> sa.ColumnElement[int]: + """Return a deterministic row number for temporal candidates.""" + return deterministic_row_number( + partition_by=tuple(partition_by), + order_by=temporal_order_expressions( + candidate_date, + anchor_date, + stable_id, + ranking, + ), + label=label, + ) diff --git a/omop_alchemy/toolkit/episodes/handling/exposure_series.py b/omop_alchemy/toolkit/episodes/handling/exposure_series.py index a7354a6..508bbce 100644 --- a/omop_alchemy/toolkit/episodes/handling/exposure_series.py +++ b/omop_alchemy/toolkit/episodes/handling/exposure_series.py @@ -9,6 +9,7 @@ def _episode_date_bounds(episode): + """Return the closed fallback window used only by opt-in recovery.""" start = episode.episode_start_date end = episode.episode_end_date or start return start, end @@ -33,6 +34,8 @@ def resolve_drug_exposure_series( seen_ids: set[int] = set() exposures: list[Drug_Exposure] = [] + # Explicit Episode_Event links are authoritative. They remain usable + # without a session and are not replaced by a broader date-based guess. for event in episode.events: if not isinstance(event, Drug_Exposure): continue @@ -43,6 +46,9 @@ def resolve_drug_exposure_series( session = object_session(episode) if include_window and session is not None: + # Date fallback is deliberately opt-in because same-person, same-date + # exposure is not sufficient evidence of episode membership. The ID + # set prevents an explicitly linked row from being returned twice. start, end = _episode_date_bounds(episode) stmt = select(Drug_Exposure).where( Drug_Exposure.person_id == episode.person_id, diff --git a/pyproject.toml b/pyproject.toml index a4e93ec..9f247ad 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -36,7 +36,8 @@ dependencies = [ "oa-configurator>=1.0.0,<2.0.0", "typer>=0.12", "rich>=13.0", - "orm-loader>=1.0.0,<2.0.0", + "orm-loader>=1.2.0,<2.0.0", + "typing-extensions>=4.5", ] [project.optional-dependencies] @@ -45,7 +46,7 @@ postgres = [ ] semantics = [ - "omop-semantics>=0.6.0", + "omop-semantics>=0.6.2", ] dev = [ diff --git a/tests/README.md b/tests/README.md index 0f70491..b19e588 100644 --- a/tests/README.md +++ b/tests/README.md @@ -4,27 +4,23 @@ ```bash # Unit and SQLite tests — no database required -uv run --extra dev pytest -m "not postgres" +uv run --extra dev pytest -m "not requires_database" -# PostgreSQL integration tests — requires the Docker container below -docker compose -f tests/docker-compose.yaml up -d -uv run --extra dev --extra postgres pytest -m postgres -v +# PostgreSQL integration tests — requires a configured test_cdm_db +uv run --extra dev --extra postgres pytest -m requires_database -v ``` ## PostgreSQL integration tests -The `postgres`-marked tests connect to a local PostgreSQL 16 container on -port **55432**. +PostgreSQL provisioning belongs to the workspace stack rather than this package; +there is no package-local Compose file. Configure its dedicated test database +using `omop-config configure omop_alchemy`. The resulting `test_cdm_db` +connection must have `test_only = true`—the test plugin rejects an ordinary +connection because these tests recreate its `public` schema. ```bash -# Start -docker compose -f tests/example-docker-compose.yaml up -d - -# Run (this will run all tests) -uv run --extra dev --extra postgres pytest -m "postgres or not postgres" -v - -# Stop -docker compose -f tests/docker-compose.yaml down +# Run the complete suite; database tests skip if test_cdm_db is absent. +uv run --extra dev --extra postgres pytest -v ``` ## Test markers @@ -32,10 +28,10 @@ docker compose -f tests/docker-compose.yaml down | Marker | Meaning | |--------|---------| | *(none)* | Runs on SQLite, no external dependencies | -| `postgres` | Requires the Docker container on port 55432 | +| `requires_database("test_cdm_db")` | Requires the configured PostgreSQL test database | ## Fixture data `tests/fixtures/athena_source/` contains a minimal set of Athena vocabulary CSVs (7 concepts) used to seed the SQLite test database. These are committed -to the repo and are sufficient for all non-postgres tests. +to the repo and are sufficient for all tests not marked `requires_database`. diff --git a/tests/fixtures/__init__.py b/tests/fixtures/__init__.py new file mode 100644 index 0000000..9d65cdb --- /dev/null +++ b/tests/fixtures/__init__.py @@ -0,0 +1 @@ +"""Small, reviewable datasets shared by semantic contract tests.""" diff --git a/tests/fixtures/query_contract_cases.py b/tests/fixtures/query_contract_cases.py new file mode 100644 index 0000000..5ec6ba6 --- /dev/null +++ b/tests/fixtures/query_contract_cases.py @@ -0,0 +1,182 @@ +"""Counterexamples that clinical query builders must satisfy. + +The IDs are intentionally small and collide across event tables. Dates are +chosen so precedence, boundaries, and ties can be verified by inspection. +This module contains data only; it does not encode the expected algorithm. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import date + +from omop_alchemy.cdm.base import ModifierFieldConcepts +from omop_alchemy.toolkit.core.events import ClinicalEventIdentity + + +@dataclass(frozen=True, slots=True) +class EventCase: + identity: ClinicalEventIdentity + person_id: int + event_date: date + event_field_concept_id: int + + +@dataclass(frozen=True, slots=True) +class EpisodeCase: + episode_id: int + person_id: int + start_date: date + end_date: date | None + + +@dataclass(frozen=True, slots=True) +class ExplicitLinkCase: + event: ClinicalEventIdentity + episode_id: int + episode_event_field_concept_id: int + + +@dataclass(frozen=True, slots=True) +class ObservationCase: + observation_id: int + person_id: int + observation_concept_id: int + observation_date: date + value: str + + +# Numeric event ID 7 exists in two event tables for the same person. A third +# event with ID 7 belongs to another person. Numeric ID alone cannot identify +# any of these rows safely. +COLLIDING_EVENTS = ( + EventCase( + ClinicalEventIdentity("measurement", 7), + person_id=101, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.MEASUREMENT, + ), + EventCase( + ClinicalEventIdentity("procedure_occurrence", 7), + person_id=101, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ), + EventCase( + ClinicalEventIdentity("observation", 7), + person_id=202, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.OBSERVATION, + ), +) + + +# Episodes 1001 and 1002 overlap. Their start dates are equally distant from +# 20 January, so nearest selection must use episode_id as its final tie-break. +OVERLAPPING_EPISODES = ( + EpisodeCase( + 1001, person_id=101, start_date=date(2026, 1, 15), end_date=date(2026, 2, 5) + ), + EpisodeCase( + 1002, person_id=101, start_date=date(2026, 1, 25), end_date=date(2026, 2, 20) + ), + EpisodeCase(2001, person_id=202, start_date=date(2026, 1, 10), end_date=None), +) + + +# For the 20 January event, episode 1003 is technically closer but has not +# started. A side-neutral nearest policy selects 1003; a policy that prefers +# already-started episodes selects 1001. +DIRECTIONAL_PREFERENCE_EPISODES = ( + EpisodeCase( + 1001, person_id=101, start_date=date(2026, 1, 15), end_date=date(2026, 2, 5) + ), + EpisodeCase( + 1003, person_id=101, start_date=date(2026, 1, 21), end_date=date(2026, 2, 28) + ), +) + + +# These future-only candidates reproduce the signed-distance pitfall in the +# current omop-constructs visit ranking. With diff = event - episode start, +# ascending signed values choose 4002 even though 4001 is closer. +CONSTRUCTS_FUTURE_VISIT_EPISODES = ( + EpisodeCase(4001, person_id=101, start_date=date(2026, 8, 1), end_date=None), + EpisodeCase(4002, person_id=101, start_date=date(2026, 9, 1), end_date=None), +) + +# Current omop-constructs uses abs(diff_days) < 180 for its first visit tier. +# This start is exactly 180 days before the event and therefore exposes the +# strict-boundary behaviour. +CONSTRUCTS_EXACT_180_DAY_EPISODE = EpisodeCase( + 4003, + person_id=101, + start_date=date(2025, 7, 24), + end_date=None, +) + + +VALID_EXPLICIT_LINK = ExplicitLinkCase( + event=ClinicalEventIdentity("procedure_occurrence", 7), + episode_id=1002, + episode_event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, +) + +COLLIDING_VALID_LINK = ExplicitLinkCase( + event=ClinicalEventIdentity("measurement", 7), + episode_id=1001, + episode_event_field_concept_id=ModifierFieldConcepts.MEASUREMENT, +) + +OUT_OF_SCOPE_LINK = ExplicitLinkCase( + event=ClinicalEventIdentity("drug_exposure", 7), + episode_id=1001, + episode_event_field_concept_id=ModifierFieldConcepts.DRUG_EXPOSURE, +) + +CROSS_PERSON_LINK = ExplicitLinkCase( + event=ClinicalEventIdentity("observation", 7), + episode_id=1001, + episode_event_field_concept_id=ModifierFieldConcepts.OBSERVATION, +) + + +# With a 90-day prior window and episode 1001 starting on 15 January, 17 +# October is the inclusive lower boundary and 16 October is outside it. +BOUNDARY_EVENTS = ( + EventCase( + ClinicalEventIdentity("measurement", 8), + person_id=101, + event_date=date(2025, 10, 17), + event_field_concept_id=ModifierFieldConcepts.MEASUREMENT, + ), + EventCase( + ClinicalEventIdentity("measurement", 9), + person_id=101, + event_date=date(2025, 10, 16), + event_field_concept_id=ModifierFieldConcepts.MEASUREMENT, + ), + EventCase( + ClinicalEventIdentity("measurement", 10), + person_id=101, + event_date=date(2026, 2, 5), + event_field_concept_id=ModifierFieldConcepts.MEASUREMENT, + ), + EventCase( + ClinicalEventIdentity("measurement", 11), + person_id=101, + event_date=date(2026, 2, 6), + event_field_concept_id=ModifierFieldConcepts.MEASUREMENT, + ), +) + + +# Two observations occur on the anchor date. Stable ascending observation_id +# makes 22 the deterministic winner for latest-on-or-before-anchor. +OBSERVATION_ANCHOR_DATE = date(2026, 1, 20) +REPEATED_OBSERVATIONS = ( + ObservationCase(21, 101, 900_001, date(2026, 1, 1), "earlier"), + ObservationCase(22, 101, 900_001, OBSERVATION_ANCHOR_DATE, "anchor-a"), + ObservationCase(23, 101, 900_001, OBSERVATION_ANCHOR_DATE, "anchor-b"), + ObservationCase(24, 101, 900_001, date(2026, 1, 21), "after-anchor"), +) diff --git a/tests/test_clinical_event_union_deprecation.py b/tests/test_clinical_event_union_deprecation.py new file mode 100644 index 0000000..ebb99d0 --- /dev/null +++ b/tests/test_clinical_event_union_deprecation.py @@ -0,0 +1,51 @@ +"""Compatibility warning for the superseded mapped clinical-event prototype.""" + +from __future__ import annotations + +import subprocess +import sys +import textwrap + + +def test_clinical_event_union_module_warns_on_direct_import(): + result = subprocess.run( + [ + sys.executable, + "-W", + "always::DeprecationWarning", + "-c", + textwrap.dedent( + """ + import warnings + import sqlalchemy as sa + from sqlalchemy.dialects import sqlite + from omop_alchemy.cdm.model.clinical.clinical_event_union import ( + ClinicalEventView, clinical_event_union, + ) + + assert "canonical_event_union" in ClinicalEventView.__deprecated__ + assert "removed in omop-alchemy 2.0" in ClinicalEventView.__deprecated__ + with warnings.catch_warnings(record=True) as emitted: + warnings.simplefilter("always", DeprecationWarning) + mapper = sa.inspect(ClinicalEventView) + assert mapper.local_table is clinical_event_union + assert [column.key for column in mapper.primary_key] == [ + "domain", "event_id", + ] + row = ClinicalEventView(domain="condition", event_id=7) + assert sa.inspect(row).mapper is mapper + class SpecializedEvent(ClinicalEventView): + pass + assert sa.inspect(SpecializedEvent).inherits is mapper + sa.select(ClinicalEventView).compile(dialect=sqlite.dialect()) + assert not emitted + """ + ), + ], + capture_output=True, + check=True, + text=True, + ) + + assert "canonical_event_union" in result.stderr + assert "removed in omop-alchemy 2.0" in result.stderr diff --git a/tests/test_concept_groups.py b/tests/test_concept_groups.py index 452f606..ee55a8d 100644 --- a/tests/test_concept_groups.py +++ b/tests/test_concept_groups.py @@ -13,6 +13,7 @@ from omop_alchemy.toolkit.core.concepts import ( ConceptGroupSpec, + SemanticUnitRef, build_concept_group, clear_concept_group_cache, concept_group_cache_stats, @@ -49,6 +50,7 @@ def _clean_cache(): # ── laziness ──────────────────────────────────────────────────────────────── + def test_importing_oncology_touches_no_database_or_semantics(): """The oncology package must import with no database and no semantics runtime. @@ -82,6 +84,25 @@ def __getattr__(self, name): ConceptGroupSpec(name="lazy", unit=Exploding()) +def test_semantic_unit_reference_is_a_lazy_complete_role_adapter(): + pytest.importorskip("omop_semantics") + from omop_semantics.runtime.default_valuesets import runtime + + ref = SemanticUnitRef("condition_modifiers", "metastatic_disease_concepts") + + assert ref.parent_ids == { + runtime.condition_modifiers.condition_modifier_values.metastatic_disease + } + assert ref.excluded_parent_ids == set() + assert ref.exact_ids == set() + + +@pytest.mark.parametrize(("value_set", "unit"), [("", "unit"), ("set", "")]) +def test_semantic_unit_reference_rejects_empty_paths(value_set: str, unit: str): + with pytest.raises(ValueError, match="must not be empty"): + SemanticUnitRef(value_set, unit) + + # ── resolution semantics ──────────────────────────────────────────────────── # The fixture vocabulary ships an empty concept_ancestor, so these tests insert # the ancestry they need. They call build_concept_group directly rather than @@ -162,6 +183,7 @@ def test_membership_rejects_none(session): # ── the two access paths agree ────────────────────────────────────────────── + def test_python_and_sql_paths_agree(session): """The instance and expression forms must select the same concepts. @@ -204,6 +226,7 @@ def test_empty_group_expression_is_false(): # ── caching ───────────────────────────────────────────────────────────────── + def test_expansion_is_cached_per_vocabulary(session): """A second request must not rebuild.""" spec = _spec(name="cached", parents=(), exact=(1,)) @@ -221,7 +244,9 @@ def test_registered_identity_is_shared_across_engines(session): try: registry_a = concept_group_registry(session) with so.Session(engine) as other_session: - register_vocabulary_identity(other_session.get_bind().engine, "test-vocab-identity") + register_vocabulary_identity( + other_session.get_bind().engine, "test-vocab-identity" + ) registry_b = concept_group_registry(other_session) assert registry_a is registry_b finally: @@ -254,11 +279,14 @@ def test_connection_bound_sessions_share_their_engine_scope(session): engine = session.get_bind().engine with engine.connect() as connection: with so.Session(bind=connection) as conn_session: - assert concept_group_registry(conn_session) is concept_group_registry(session) + assert concept_group_registry(conn_session) is concept_group_registry( + session + ) # ── bounded cache and its observability ───────────────────────────────────── + def test_eviction_is_bounded_and_counted(session): """A too-small bound must evict, and a rebuild after eviction must be counted. @@ -297,12 +325,13 @@ def test_cache_stats_are_reportable(session): assert all("rebuilds_after_evict" in v for v in stats.values()) -# ── governed specs use the non-deprecated 0.6.0 surface ───────────────────── +# ── governed specs use the non-deprecated 0.6+ surface ────────────────────── + def test_governed_specs_emit_no_deprecation_warnings(): - """The oncology specs must read 0.6.0's role-specific accessors. + """The oncology specs must read the 0.6+ role-specific accessors. - Guards against slipping back to group-backed `.ids`, which 0.6.0 deprecates + Guards against slipping back to group-backed `.ids`, which 0.6 deprecates because it does not say whether members expand through descendants. """ pytest.importorskip("omop_semantics") @@ -347,13 +376,12 @@ def test_radiotherapy_spec_matches_governed_group(): # ConceptFilter: require_standard / include_classification. These pin that the # names mean the same thing here as everywhere else in the package. + def _rendered(spec) -> str: """Compile the group's membership predicate to inspectable SQL.""" column = sa.column("concept_id") return str( - spec.expression_for(column).compile( - compile_kwargs={"literal_binds": True} - ) + spec.expression_for(column).compile(compile_kwargs={"literal_binds": True}) ).lower() diff --git a/tests/test_concept_mapping_queries.py b/tests/test_concept_mapping_queries.py new file mode 100644 index 0000000..59d83f5 --- /dev/null +++ b/tests/test_concept_mapping_queries.py @@ -0,0 +1,209 @@ +"""Standard concept mapping query contracts.""" + +from __future__ import annotations + +from datetime import date + +import pytest +from sqlalchemy.dialects import postgresql, sqlite + +from omop_alchemy.cdm.model.vocabulary import Concept, Concept_Relationship +from omop_alchemy.toolkit.core.concepts import ( + STANDARD_CONCEPT_MAPPING_UNIQUENESS, + StandardConceptMappingSpec, + standard_concept_mapping_select, +) + + +SOURCE_ID = 991_000 +STANDARD_TARGET_ID = 991_001 +SECOND_STANDARD_TARGET_ID = 991_002 +NON_STANDARD_TARGET_ID = 991_003 +INVALID_TARGET_ID = 991_004 +STANDARD_SOURCE_ID = 991_005 + + +def _concept( + concept_id: int, + name: str, + *, + standard_concept: str | None, + invalid_reason: str | None = None, +) -> Concept: + return Concept( + concept_id=concept_id, + concept_name=name, + domain_id="Condition", + vocabulary_id="SNOMED", + concept_class_id="Clinical Finding", + standard_concept=standard_concept, + concept_code=f"code-{concept_id}", + valid_start_date=date(2000, 1, 1), + valid_end_date=date(2099, 12, 31), + invalid_reason=invalid_reason, + ) + + +def _relationship( + source_id: int, + target_id: int, + relationship_id: str = "Maps to", + *, + valid_end_date: date = date(2099, 12, 31), + invalid_reason: str | None = None, +) -> Concept_Relationship: + return Concept_Relationship( + concept_id_1=source_id, + concept_id_2=target_id, + relationship_id=relationship_id, + valid_start_date=date(2000, 1, 1), + valid_end_date=valid_end_date, + invalid_reason=invalid_reason, + ) + + +def _add_vocabulary_rows(session) -> None: + session.add_all( + ( + _concept(SOURCE_ID, "source", standard_concept=None), + _concept(STANDARD_TARGET_ID, "first standard", standard_concept="S"), + _concept( + SECOND_STANDARD_TARGET_ID, + "second standard", + standard_concept="S", + ), + _concept(NON_STANDARD_TARGET_ID, "non-standard", standard_concept=None), + _concept( + INVALID_TARGET_ID, + "invalid standard", + standard_concept="S", + invalid_reason="D", + ), + _concept(STANDARD_SOURCE_ID, "standard source", standard_concept="S"), + ) + ) + session.flush() + session.add_all( + ( + _relationship(SOURCE_ID, STANDARD_TARGET_ID), + _relationship(SOURCE_ID, SECOND_STANDARD_TARGET_ID), + _relationship(SOURCE_ID, NON_STANDARD_TARGET_ID), + _relationship(SOURCE_ID, INVALID_TARGET_ID), + _relationship( + SOURCE_ID, + STANDARD_SOURCE_ID, + valid_end_date=date(2020, 12, 31), + invalid_reason="U", + ), + _relationship(SOURCE_ID, STANDARD_TARGET_ID, "Maps to value"), + _relationship(STANDARD_SOURCE_ID, STANDARD_SOURCE_ID), + ) + ) + session.flush() + + +def test_mapping_preserves_multiple_standard_targets(session): + _add_vocabulary_rows(session) + + rows = ( + session.execute( + standard_concept_mapping_select( + StandardConceptMappingSpec(source_concept_ids=(SOURCE_ID,)) + ) + ) + .mappings() + .all() + ) + + assert {row["standard_concept_id"] for row in rows} == { + STANDARD_TARGET_ID, + SECOND_STANDARD_TARGET_ID, + } + assert rows[0]["source_concept_name"] == "source" + assert rows[0]["source_vocabulary_id"] == "SNOMED" + assert rows[0]["standard_vocabulary_id"] == "SNOMED" + + +def test_mapping_excludes_other_relationships_and_invalid_or_non_standard_targets( + session, +): + _add_vocabulary_rows(session) + + rows = ( + session.execute( + standard_concept_mapping_select( + StandardConceptMappingSpec(source_concept_ids=(SOURCE_ID,)) + ) + ) + .mappings() + .all() + ) + + assert len(rows) == 2 + + +def test_mapping_returns_standard_concept_self_map(session): + _add_vocabulary_rows(session) + + row = ( + session.execute( + standard_concept_mapping_select( + StandardConceptMappingSpec(source_concept_ids=(STANDARD_SOURCE_ID,)) + ) + ) + .mappings() + .one() + ) + + assert row["source_concept_id"] == STANDARD_SOURCE_ID + assert row["standard_concept_id"] == STANDARD_SOURCE_ID + + +def test_mapping_can_apply_reproducible_validity_date(session): + _add_vocabulary_rows(session) + + rows = ( + session.execute( + standard_concept_mapping_select( + StandardConceptMappingSpec( + source_concept_ids=(SOURCE_ID,), + valid_on=date(2026, 1, 1), + ) + ) + ) + .mappings() + .all() + ) + + assert {row["standard_concept_id"] for row in rows} == { + STANDARD_TARGET_ID, + SECOND_STANDARD_TARGET_ID, + } + + +def test_mapping_output_honours_its_documented_uniqueness_key(session): + _add_vocabulary_rows(session) + rows = ( + session.execute(standard_concept_mapping_select(StandardConceptMappingSpec())) + .mappings() + .all() + ) + key_names = tuple(str(column) for column in STANDARD_CONCEPT_MAPPING_UNIQUENESS) + keys = {tuple(row[name] for name in key_names) for row in rows} + + assert len(keys) == len(rows) + + +@pytest.mark.parametrize("dialect", [sqlite.dialect(), postgresql.dialect()]) +def test_mapping_query_compiles_on_supported_dialects(dialect): + statement = standard_concept_mapping_select( + StandardConceptMappingSpec( + source_concept_ids=(SOURCE_ID,), + valid_on=date(2026, 1, 1), + ) + ) + + compiled = str(statement.compile(dialect=dialect)) + assert "concept_relationship" in compiled + assert "mapping_source_concept" in compiled + assert "mapping_standard_concept" in compiled diff --git a/tests/test_episode_attachment_queries.py b/tests/test_episode_attachment_queries.py new file mode 100644 index 0000000..0285fab --- /dev/null +++ b/tests/test_episode_attachment_queries.py @@ -0,0 +1,496 @@ +"""Explicit-first event-to-episode attachment queries.""" + +from __future__ import annotations + +from datetime import date + +import pytest +import sqlalchemy as sa +from sqlalchemy.dialects import postgresql, sqlite + +from omop_alchemy.cdm.base import ModifierFieldConcepts +from omop_alchemy.cdm.model import Procedure_Occurrence +from omop_alchemy.toolkit.core.events import ClinicalEventIdentity +from omop_alchemy.toolkit.episodes.derivation import ( + AttachmentDiagnosticCode, + EpisodeAttachmentDiagnostic, + EpisodeAttachmentPolicy, + EpisodeWindowSpec, + TemporalRankingSpec, + TemporalSelectionPolicy, + TemporalSidePreference, + episode_attachment_queries, +) +from tests.fixtures.query_contract_cases import ( + COLLIDING_EVENTS, + CROSS_PERSON_LINK, + DIRECTIONAL_PREFERENCE_EPISODES, + OVERLAPPING_EPISODES, + OUT_OF_SCOPE_LINK, + VALID_EXPLICIT_LINK, + COLLIDING_VALID_LINK, + EpisodeCase, + EventCase, + ExplicitLinkCase, +) + + +def _event_source(*events: EventCase) -> sa.CTE: + return sa.union_all( + *( + sa.select( + sa.literal(event.person_id).label("person_id"), + sa.literal(event.identity.event_id).label("event_id"), + sa.literal(event.event_date).label("event_date"), + sa.cast(sa.null(), sa.DateTime()).label("event_datetime"), + sa.literal(900_001).label("event_concept_id"), + sa.literal(event.event_field_concept_id).label( + "event_field_concept_id" + ), + sa.literal(event.identity.event_source_table).label( + "event_source_table" + ), + ) + for event in events + ) + ).cte("events") + + +def _episode_source(*episodes: EpisodeCase) -> sa.CTE: + return sa.union_all( + *( + sa.select( + sa.literal(episode.episode_id).label("episode_id"), + sa.literal(episode.person_id).label("person_id"), + sa.literal(episode.start_date).label("episode_start_date"), + sa.literal(episode.end_date, type_=sa.Date()).label("episode_end_date"), + ) + for episode in episodes + ) + ).cte("episodes") + + +def _link_source(*links: ExplicitLinkCase) -> sa.CTE: + return sa.union_all( + *( + sa.select( + sa.literal(link.episode_id).label("episode_id"), + sa.literal(link.event.event_id).label("event_id"), + sa.literal(link.episode_event_field_concept_id).label( + "episode_event_field_concept_id" + ), + ) + for link in links + ) + ).cte("episode_events") + + +def _empty_link_source() -> sa.CTE: + return ( + sa.select( + sa.cast(sa.null(), sa.Integer()).label("episode_id"), + sa.cast(sa.null(), sa.Integer()).label("event_id"), + sa.cast(sa.null(), sa.Integer()).label("episode_event_field_concept_id"), + ) + .where(sa.false()) + .cte("episode_events") + ) + + +def _nearest(*, started_first: bool = False) -> TemporalRankingSpec: + return TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + side_preference=( + TemporalSidePreference.on_or_before_anchor + if started_first + else TemporalSidePreference.none + ), + ) + + +def test_valid_explicit_links_suppress_ranked_fallback_with_colliding_ids(session): + unlinked = EventCase( + identity=ClinicalEventIdentity("procedure_occurrence", 8), + person_id=101, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ) + sources = episode_attachment_queries( + _event_source(*COLLIDING_EVENTS, unlinked), + episodes=_episode_source(*OVERLAPPING_EPISODES), + episode_events=_link_source( + VALID_EXPLICIT_LINK, + COLLIDING_VALID_LINK, + CROSS_PERSON_LINK, + ), + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ranking=_nearest(), + ) + + rows = session.execute(sources.attachments).mappings().all() + identities = { + (row["event_source_table"], row["event_id"], row["episode_id"]) for row in rows + } + + assert len(identities) == len(rows) + assert ("measurement", 7, 1001) in identities + assert ("procedure_occurrence", 7, 1002) in identities + assert ("procedure_occurrence", 7, 1001) not in identities + assert ("observation", 7, 2001) in identities + assert ("procedure_occurrence", 8, 1001) in identities + + +def test_invalid_explicit_link_does_not_suppress_fallback(session): + event = COLLIDING_EVENTS[2] + sources = episode_attachment_queries( + _event_source(event), + episodes=_episode_source(*OVERLAPPING_EPISODES), + episode_events=_link_source(CROSS_PERSON_LINK), + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ranking=_nearest(), + ) + + assert session.execute(sources.attachments).mappings().one()["episode_id"] == 2001 + + +def test_foreign_discriminator_link_is_out_of_scope_for_a_single_model(session): + queries = episode_attachment_queries( + _event_source(COLLIDING_EVENTS[0]), + episodes=_episode_source(*OVERLAPPING_EPISODES[:2]), + episode_events=_link_source(COLLIDING_VALID_LINK, VALID_EXPLICIT_LINK), + policy=EpisodeAttachmentPolicy.explicit_only, + include_diagnostics=True, + ) + + assert session.execute(queries.attachments).mappings().one()["episode_id"] == 1001 + assert queries.diagnostics is not None + assert session.execute(queries.diagnostics).all() == [] + + +def test_explicit_only_returns_valid_links_without_fallback(session): + queries = episode_attachment_queries( + _event_source(*COLLIDING_EVENTS), + episodes=_episode_source(*OVERLAPPING_EPISODES), + episode_events=_link_source( + VALID_EXPLICIT_LINK, + COLLIDING_VALID_LINK, + CROSS_PERSON_LINK, + ), + policy=EpisodeAttachmentPolicy.explicit_only, + ) + + rows = session.execute(queries.attachments).mappings().all() + + assert queries.diagnostics is None + assert { + (row["event_source_table"], row["event_id"], row["episode_id"]) for row in rows + } == { + ("measurement", 7, 1001), + ("procedure_occurrence", 7, 1002), + } + + +def test_side_preference_is_applied_to_ranked_fallback(session): + event = EventCase( + identity=ClinicalEventIdentity("procedure_occurrence", 8), + person_id=101, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ) + + def selected_episode(ranking: TemporalRankingSpec) -> int: + queries = episode_attachment_queries( + _event_source(event), + episodes=_episode_source(*DIRECTIONAL_PREFERENCE_EPISODES), + episode_events=_empty_link_source(), + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ranking=ranking, + ) + attachments = queries.attachments.subquery() + value = session.scalar(sa.select(attachments.c.episode_id)) + assert value is not None + return value + + assert selected_episode(_nearest()) == 1003 + assert selected_episode(_nearest(started_first=True)) == 1001 + + +def test_all_in_window_fallback_retains_each_eligible_episode(session): + event = EventCase( + identity=ClinicalEventIdentity("procedure_occurrence", 8), + person_id=101, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ) + queries = episode_attachment_queries( + _event_source(event), + episodes=_episode_source(*OVERLAPPING_EPISODES[:2]), + episode_events=_empty_link_source(), + policy=EpisodeAttachmentPolicy.explicit_first_all_in_window, + ) + attachments = queries.attachments.subquery() + + assert set(session.scalars(sa.select(attachments.c.episode_id))) == { + 1001, + 1002, + } + + +def test_all_in_window_uses_a_window_contract_without_ranking(session): + boundary_event = EventCase( + identity=ClinicalEventIdentity("procedure_occurrence", 8), + person_id=101, + event_date=date(2025, 10, 17), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ) + queries = episode_attachment_queries( + _event_source(boundary_event), + episodes=_episode_source(OVERLAPPING_EPISODES[0]), + episode_events=_empty_link_source(), + policy=EpisodeAttachmentPolicy.explicit_first_all_in_window, + window=EpisodeWindowSpec(include_lower_bound=False), + ) + + assert session.execute(queries.attachments).all() == [] + + +def test_all_in_window_diagnostics_do_not_report_intended_fanout(session): + event = EventCase( + identity=ClinicalEventIdentity("procedure_occurrence", 8), + person_id=101, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ) + queries = episode_attachment_queries( + _event_source(event), + episodes=_episode_source(*OVERLAPPING_EPISODES[:2]), + episode_events=_empty_link_source(), + policy=EpisodeAttachmentPolicy.explicit_first_all_in_window, + include_diagnostics=True, + ) + + assert queries.diagnostics is not None + assert session.execute(queries.diagnostics).mappings().all() == [] + + +@pytest.mark.parametrize( + "policy", + [ + EpisodeAttachmentPolicy.explicit_only, + EpisodeAttachmentPolicy.explicit_first_all_in_window, + ], +) +def test_non_ranked_attachment_policies_reject_ranking(policy): + with pytest.raises(ValueError, match="does not use a temporal ranking"): + episode_attachment_queries( + _event_source(COLLIDING_EVENTS[0]), + episodes=_episode_source(OVERLAPPING_EPISODES[0]), + episode_events=_empty_link_source(), + policy=policy, + ranking=_nearest(), + ) + + +def test_diagnostics_explain_person_mismatches_and_fallback_outcomes(session): + ambiguous = EventCase( + identity=ClinicalEventIdentity("procedure_occurrence", 8), + person_id=101, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ) + unlinked = EventCase( + identity=ClinicalEventIdentity("procedure_occurrence", 9), + person_id=303, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ) + queries = episode_attachment_queries( + _event_source(*COLLIDING_EVENTS, ambiguous, unlinked), + episodes=_episode_source(*OVERLAPPING_EPISODES), + episode_events=_link_source( + VALID_EXPLICIT_LINK, + COLLIDING_VALID_LINK, + OUT_OF_SCOPE_LINK, + CROSS_PERSON_LINK, + ), + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ranking=_nearest(), + include_diagnostics=True, + ) + assert queries.diagnostics is not None + + rows = session.execute(queries.diagnostics).mappings().all() + codes = {row["diagnostic_code"] for row in rows} + ambiguous_rows = [ + row + for row in rows + if row["diagnostic_code"] == str(AttachmentDiagnosticCode.ambiguous_fallback) + and row["event_id"] == 8 + ] + + assert str(AttachmentDiagnosticCode.person_mismatch) in codes + assert str(AttachmentDiagnosticCode.no_candidate_episode) in codes + assert len(ambiguous_rows) == 1 + assert ambiguous_rows[0]["candidate_count"] == 2 + person_rows = [ + row + for row in rows + if row["diagnostic_code"] == str(AttachmentDiagnosticCode.person_mismatch) + ] + typed = EpisodeAttachmentDiagnostic.from_mapping(person_rows[0]) + assert typed.code is AttachmentDiagnosticCode.person_mismatch + assert typed.event.event_id == 7 + assert ( + typed.linked_event_field_concept_id + == CROSS_PERSON_LINK.episode_event_field_concept_id + ) + assert typed.episode_id == CROSS_PERSON_LINK.episode_id + + +@pytest.mark.parametrize( + "session_fixture", + [ + "session", + pytest.param("pg_session", marks=pytest.mark.requires_database("test_cdm_db")), + ], +) +@pytest.mark.parametrize("copies", [(2, 1), (1, 2), (2, 2)]) +@pytest.mark.parametrize("episode_count", [1, 2]) +@pytest.mark.parametrize( + "policy", + [ + EpisodeAttachmentPolicy.explicit_first_ranked, + EpisodeAttachmentPolicy.explicit_first_all_in_window, + ], +) +def test_fallback_counts_distinct_episodes_with_duplicate_inputs( + request, session_fixture, copies, episode_count, policy +): + session = request.getfixturevalue(session_fixture) + event_copies, episode_copies = copies + queries = episode_attachment_queries( + _event_source(*([COLLIDING_EVENTS[0]] * event_copies)), + episodes=_episode_source( + *(OVERLAPPING_EPISODES[:episode_count] * episode_copies) + ), + episode_events=_empty_link_source(), + policy=policy, + ranking=_nearest() if policy.requires_fallback_ranking else None, + include_diagnostics=True, + ) + rows = session.execute(queries.attachments).mappings().all() + expected_ids = ( + {1001} + if policy.requires_fallback_ranking + else {episode.episode_id for episode in OVERLAPPING_EPISODES[:episode_count]} + ) + assert len(rows) == len(expected_ids) + assert {row["episode_id"] for row in rows} == expected_ids + assert queries.diagnostics is not None + diagnostics = session.execute(queries.diagnostics).mappings().all() + if episode_count == 2 and policy.requires_fallback_ranking: + assert len(diagnostics) == 1 + assert diagnostics[0]["diagnostic_code"] == "ambiguous_fallback" + assert diagnostics[0]["candidate_count"] == 2 + else: + assert diagnostics == [] + + +def test_diagnostics_and_fallback_share_the_explicit_event_key_cte(): + queries = episode_attachment_queries( + _event_source(COLLIDING_EVENTS[0]), + episodes=_episode_source(*OVERLAPPING_EPISODES), + episode_events=_empty_link_source(), + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ranking=_nearest(), + include_diagnostics=True, + ) + + assert queries.diagnostics is not None + compiled = str(queries.diagnostics.compile(dialect=postgresql.dialect())) + + assert "valid_explicit_event_keys_for_fallback" not in compiled + assert compiled.count("valid_explicit_event_keys AS") == 1 + + +def test_ranked_policy_requires_a_ranking_contract(): + with pytest.raises(ValueError, match="requires a temporal ranking"): + episode_attachment_queries( + _event_source(COLLIDING_EVENTS[0]), + episodes=_episode_source(OVERLAPPING_EPISODES[0]), + episode_events=_link_source(OUT_OF_SCOPE_LINK), + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ) + + +@pytest.mark.parametrize("dialect", [sqlite.dialect(), postgresql.dialect()]) +def test_attachment_and_diagnostics_compile_on_supported_dialects(dialect): + queries = episode_attachment_queries( + _event_source(*COLLIDING_EVENTS), + episodes=_episode_source(*OVERLAPPING_EPISODES), + episode_events=_link_source( + VALID_EXPLICIT_LINK, + COLLIDING_VALID_LINK, + CROSS_PERSON_LINK, + ), + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ranking=_nearest(), + include_diagnostics=True, + ) + + str(queries.attachments.compile(dialect=dialect)) + assert queries.diagnostics is not None + str(queries.diagnostics.compile(dialect=dialect)) + + +def test_attachment_builder_accepts_a_supported_event_model(): + queries = episode_attachment_queries( + Procedure_Occurrence, + policy=EpisodeAttachmentPolicy.explicit_only, + ) + + assert "procedure_occurrence" in str( + queries.attachments.compile(dialect=postgresql.dialect()) + ) + + +@pytest.mark.requires_database("test_cdm_db") +def test_postgresql_executes_collision_and_stable_tie_contracts(pg_session): + unlinked = EventCase( + identity=ClinicalEventIdentity("procedure_occurrence", 8), + person_id=101, + event_date=date(2026, 1, 20), + event_field_concept_id=ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ) + queries = episode_attachment_queries( + _event_source(*COLLIDING_EVENTS[:2], unlinked), + episodes=_episode_source(*OVERLAPPING_EPISODES[:2]), + episode_events=_link_source(VALID_EXPLICIT_LINK, COLLIDING_VALID_LINK), + policy=EpisodeAttachmentPolicy.explicit_first_ranked, + ranking=_nearest(), + include_diagnostics=True, + ) + + rows = pg_session.execute(queries.attachments).mappings().all() + assert { + (row["event_source_table"], row["event_id"], row["episode_id"]) for row in rows + } == { + ("measurement", 7, 1001), + ("procedure_occurrence", 7, 1002), + ("procedure_occurrence", 8, 1001), + } + + single_model = episode_attachment_queries( + _event_source(COLLIDING_EVENTS[0]), + episodes=_episode_source(*OVERLAPPING_EPISODES[:2]), + episode_events=_link_source(COLLIDING_VALID_LINK, VALID_EXPLICIT_LINK), + policy=EpisodeAttachmentPolicy.explicit_only, + include_diagnostics=True, + ) + assert ( + pg_session.execute(single_model.attachments).mappings().one()["episode_id"] + == 1001 + ) + assert single_model.diagnostics is not None + assert pg_session.execute(single_model.diagnostics).all() == [] diff --git a/tests/test_episode_structure_queries.py b/tests/test_episode_structure_queries.py new file mode 100644 index 0000000..5b246c6 --- /dev/null +++ b/tests/test_episode_structure_queries.py @@ -0,0 +1,205 @@ +"""Canonical episode and hierarchy projections.""" + +from __future__ import annotations + +from datetime import date + +import sqlalchemy as sa +from sqlalchemy.dialects import postgresql, sqlite + +from omop_alchemy.cdm.model.structural import Episode, Episode_Event +from omop_alchemy.toolkit.episodes.derivation import ( + CANONICAL_EPISODE_COLUMNS, + CANONICAL_EPISODE_OPTIONAL_COLUMNS, + canonical_episode_projection, + direct_episode_relationship_projection, + episode_descendants, + episode_event_hierarchy_projection, +) + + +def test_canonical_episode_projection_has_stable_fields(): + statement = canonical_episode_projection() + + assert tuple(statement.selected_columns.keys()) == tuple( + map(str, CANONICAL_EPISODE_COLUMNS) + ) + + +def test_canonical_episode_projection_can_include_the_episode_concept_name(session): + source = session.get(Episode, 100) + assert source is not None + session.add( + Episode( + episode_id=102, + person_id=source.person_id, + episode_start_date=date(2020, 2, 1), + episode_end_date=None, + episode_concept_id=0, + episode_object_concept_id=source.episode_object_concept_id, + episode_type_concept_id=source.episode_type_concept_id, + ) + ) + session.flush() + + unlabelled = canonical_episode_projection().where( + sa.column("episode_id").in_((100, 102)) + ) + labelled = canonical_episode_projection(include_concept_label=True).where( + sa.column("episode_id").in_((100, 102)) + ) + unlabelled_rows = session.execute(unlabelled).mappings().all() + labelled_rows = session.execute(labelled).mappings().all() + + assert tuple(labelled.selected_columns.keys()) == tuple( + map(str, CANONICAL_EPISODE_COLUMNS + CANONICAL_EPISODE_OPTIONAL_COLUMNS) + ) + assert ( + {row["episode_id"] for row in labelled_rows} + == {row["episode_id"] for row in unlabelled_rows} + == {100, 102} + ) + labels = {row["episode_id"]: row["episode_concept_name"] for row in labelled_rows} + assert labels == {100: "Disease Episode", 102: None} + + +def test_hierarchy_projections_accept_a_filtered_projected_episode_source(session): + source = canonical_episode_projection().where(Episode.person_id == 1) + + direct_rows = ( + session.execute(direct_episode_relationship_projection(episode_model=source)) + .mappings() + .all() + ) + assert direct_rows == [ + { + "root_episode_id": 100, + "episode_id": 101, + "episode_parent_id": 100, + "person_id": 1, + "depth": 1, + } + ] + + hierarchy = episode_descendants( + root_episode_id=100, + episode_model=source, + ) + rows = session.execute( + sa.select(hierarchy.c.episode_id, hierarchy.c.depth).order_by( + hierarchy.c.depth, + hierarchy.c.episode_id, + ) + ).all() + assert rows == [(100, 0), (101, 1)] + + +def test_episode_event_projection_accepts_projected_episode_and_event_sources( + session, +): + episode_source = canonical_episode_projection().where(Episode.person_id == 1) + event_source = sa.select(Episode_Event.__table__).subquery("episode_events") + + rows = ( + session.execute( + episode_event_hierarchy_projection( + root_episode_id=100, + episode_model=episode_source, + episode_event_model=event_source, + ) + ) + .mappings() + .all() + ) + + assert rows == [ + { + "root_episode_id": 100, + "linked_episode_id": 101, + "person_id": 1, + "episode_depth": 1, + "event_id": 1, + "event_field_concept_id": 1147127, + } + ] + + +def test_episode_concept_label_projection_compiles_on_supported_dialects(): + for dialect in (sqlite.dialect(), postgresql.dialect()): + compiled = str( + canonical_episode_projection(include_concept_label=True).compile( + dialect=dialect + ) + ) + assert "LEFT OUTER JOIN" in compiled + + +def test_direct_episode_relationships_preserve_person_and_depth(session): + rows = ( + session.execute( + direct_episode_relationship_projection().where( + sa.column("root_episode_id") == 100 + ) + ) + .mappings() + .all() + ) + + assert rows == [ + { + "root_episode_id": 100, + "episode_id": 101, + "episode_parent_id": 100, + "person_id": 1, + "depth": 1, + } + ] + + +def test_recursive_episode_projection_includes_root_and_descendants(session): + hierarchy = episode_descendants(root_episode_id=100) + rows = session.execute( + sa.select(hierarchy.c.episode_id, hierarchy.c.depth).order_by( + hierarchy.c.depth, + hierarchy.c.episode_id, + ) + ).all() + + assert rows == [(100, 0), (101, 1)] + + +def test_recursive_episode_projection_can_exclude_root(session): + hierarchy = episode_descendants(root_episode_id=100, include_root=False) + + assert session.execute( + sa.select(hierarchy.c.episode_id, hierarchy.c.depth) + ).all() == [(101, 1)] + + +def test_episode_event_projection_carries_owning_depth(session): + rows = ( + session.execute(episode_event_hierarchy_projection(root_episode_id=100)) + .mappings() + .all() + ) + + assert rows == [ + { + "root_episode_id": 100, + "linked_episode_id": 101, + "person_id": 1, + "episode_depth": 1, + "event_id": 1, + "event_field_concept_id": 1147127, + } + ] + + +def test_recursive_episode_projection_compiles_on_supported_dialects(): + hierarchy = episode_descendants(root_episode_id=100) + statement = sa.select(*hierarchy.c) + + for dialect in (sqlite.dialect(), postgresql.dialect()): + compiled = str(statement.compile(dialect=dialect)) + assert "WITH RECURSIVE" in compiled + assert "episode_parent_id" in compiled diff --git a/tests/test_episodes_basic.py b/tests/test_episodes_basic.py index 7d3e7b9..9bf3ed8 100644 --- a/tests/test_episodes_basic.py +++ b/tests/test_episodes_basic.py @@ -30,7 +30,9 @@ def test_episode_view_expected_domains(): assert "episode_object_concept_id" in cls.__expected_domains__ assert "episode_type_concept_id" in cls.__expected_domains__ - assert cls.__expected_domains__["episode_concept_id"].domains == frozenset({"Episode"}) + assert cls.__expected_domains__["episode_concept_id"].domains == frozenset( + {"Episode"} + ) def test_episode_reference_context(session): @@ -46,11 +48,7 @@ def test_episode_reference_context(session): def test_episode_has_episode_events(session): """Test episode has episode events.""" - ep = ( - session.query(EpisodeView) - .filter(EpisodeView.episode_events.any()) - .first() - ) + ep = session.query(EpisodeView).filter(EpisodeView.episode_events.any()).first() assert ep is not None assert len(ep.episode_events) > 0 @@ -73,11 +71,7 @@ def test_resolved_episode_event_mixin_targets_diagnostic_rows(session): def test_episode_event_resolves_target(session): """Test episode event resolves target.""" - ep = ( - session.query(EpisodeView) - .filter(EpisodeView.episode_events.any()) - .first() - ) + ep = session.query(EpisodeView).filter(EpisodeView.episode_events.any()).first() ee = ep.episode_events[0] target = ee.resolved_event @@ -185,28 +179,22 @@ def test_episode_event_resolution_reports_dangling_target(session): assert diagnostics[0].kind == "dangling_event" -def test_episode_event_target_class_cache_can_be_cleared(): - """The resolver map is memoized but explicitly invalidatable.""" - clear_episode_event_target_class_cache() +def test_episode_event_target_classes_are_isolated_from_caller_mutation(): + """Resolver overrides cannot mutate the stable Core target registry.""" first = Episode_EventView.resolved_event_target_classes() second = Episode_EventView.resolved_event_target_classes() - assert first is second - + assert first is not second + first.clear() clear_episode_event_target_class_cache() third = Episode_EventView.resolved_event_target_classes() - assert third is not first - assert third == first + assert third == second def test_episode_view_events_property(session): """Test episode view events property.""" - ep = ( - session.query(EpisodeView) - .filter(EpisodeView.episode_events.any()) - .first() - ) + ep = session.query(EpisodeView).filter(EpisodeView.episode_events.any()).first() events = ep.events @@ -221,7 +209,6 @@ def test_episode_view_events_property(session): assert hasattr(target, "person_id") - def test_episode_parent_relationship(session): """Test episode parent relationship.""" child = ( diff --git a/tests/test_event_projections.py b/tests/test_event_projections.py new file mode 100644 index 0000000..6524c37 --- /dev/null +++ b/tests/test_event_projections.py @@ -0,0 +1,314 @@ +"""Canonical clinical-event projection behaviour.""" + +from __future__ import annotations + +import subprocess +import sys +import textwrap + +import pytest +from sqlalchemy.dialects import postgresql, sqlite + +from omop_alchemy.cdm.base import ModifierFieldConcepts +from omop_alchemy.cdm.model import ( + Condition_Occurrence, + Device_Exposure, + Drug_Exposure, + Measurement, + Observation, + Person, + Procedure_Occurrence, +) +from omop_alchemy.cdm.model.clinical import ( + Condition_OccurrenceView, + Device_ExposureView, + Drug_ExposureView, + MeasurementView, + ObservationView, + Procedure_OccurrenceView, +) +from omop_alchemy.cdm.base import ClinicalEventMixin, ModifierTargetMixin +from omop_alchemy.cdm.base.event_metadata import ( + UnsupportedClinicalEventModelError, +) +from omop_alchemy.cdm.model.structural import Episode, Episode_EventView, EpisodeView +from omop_alchemy.cdm.model.clinical.event_metadata import ( + _validate_unique_target_keys, + clinical_event_model_spec, + clinical_event_target_for_table, +) +from omop_alchemy.toolkit.core.events import ( + ClinicalEventColumn, + canonical_event_projection, + canonical_event_union, +) + + +@pytest.mark.parametrize( + ("model", "source_table", "field_concept_id"), + [ + ( + Condition_Occurrence, + "condition_occurrence", + ModifierFieldConcepts.CONDITION_OCCURRENCE, + ), + ( + Device_Exposure, + "device_exposure", + ModifierFieldConcepts.DEVICE_EXPOSURE, + ), + (Drug_Exposure, "drug_exposure", ModifierFieldConcepts.DRUG_EXPOSURE), + (Measurement, "measurement", ModifierFieldConcepts.MEASUREMENT), + (Observation, "observation", ModifierFieldConcepts.OBSERVATION), + ( + Procedure_Occurrence, + "procedure_occurrence", + ModifierFieldConcepts.PROCEDURE_OCCURRENCE, + ), + ], +) +def test_projection_resolves_source_metadata( + model, + source_table: str, + field_concept_id: int, +): + spec = clinical_event_model_spec(model) + view = clinical_event_target_for_table(source_table) + assert issubclass(view, ClinicalEventMixin) + assert view.has_complete_metadata() + assert view.clinical_event_model_spec(model) == spec + assert view.clinical_event_model_spec() == spec + statement = canonical_event_projection(model) + + assert spec.event_source_table == source_table + assert spec.event_field_concept_id == field_concept_id + assert tuple(statement.selected_columns.keys()) == tuple( + map( + str, + ClinicalEventColumn.required_columns() + + ClinicalEventColumn.optional_columns(), + ) + ) + + +@pytest.mark.parametrize("dialect", [sqlite.dialect(), postgresql.dialect()]) +def test_projection_compiles_discriminator_and_source_as_literals(dialect): + statement = canonical_event_projection(Measurement) + compiled = str( + statement.compile(dialect=dialect, compile_kwargs={"literal_binds": True}) + ) + + assert str(ModifierFieldConcepts.MEASUREMENT) in compiled + assert "'measurement'" in compiled + assert "measurement.value_as_number AS value_as_number" in compiled + + +def test_non_value_event_projects_typed_null_value_columns(): + compiled = str( + canonical_event_projection(Procedure_Occurrence).compile( + dialect=postgresql.dialect(), + compile_kwargs={"literal_binds": True}, + ) + ) + + assert "CAST(NULL AS FLOAT) AS value_as_number" in compiled + assert "CAST(NULL AS INTEGER) AS value_as_concept_id" in compiled + + +def test_projection_union_preserves_one_shared_shape(): + statement = canonical_event_union( + Measurement, + Observation, + Procedure_Occurrence, + ) + compiled = str(statement.compile(dialect=sqlite.dialect())) + + assert tuple(statement.selected_columns.keys()) == tuple( + map( + str, + ClinicalEventColumn.required_columns() + + ClinicalEventColumn.optional_columns(), + ) + ) + assert compiled.count("UNION ALL") == 2 + + +def test_incomplete_modifier_target_has_a_typed_error(): + with pytest.raises( + UnsupportedClinicalEventModelError, + match="no complete ClinicalEventMixin metadata", + ) as raised: + canonical_event_projection(Person) + + assert raised.value.model is Person + assert raised.value.reason == "no complete ClinicalEventMixin metadata is available" + + +@pytest.mark.parametrize("model", [Episode, EpisodeView]) +def test_structural_modifier_targets_are_not_clinical_events(model): + with pytest.raises( + UnsupportedClinicalEventModelError, + match="no complete ClinicalEventMixin metadata", + ): + clinical_event_model_spec(model) + + +def test_target_registry_rejects_duplicate_identities(): + entries = ( + (Condition_Occurrence, Condition_OccurrenceView), + (Measurement, MeasurementView), + ) + + with pytest.raises(ValueError, match="duplicate test identity"): + _validate_unique_target_keys( + entries, + key=lambda _source, _target: "same", + label="test identity", + ) + + +def test_all_core_event_views_are_registered_episode_event_targets(): + targets = Episode_EventView.resolved_event_target_classes() + + expected = { + ModifierFieldConcepts.CONDITION_OCCURRENCE: Condition_OccurrenceView, + ModifierFieldConcepts.DEVICE_EXPOSURE: Device_ExposureView, + ModifierFieldConcepts.DRUG_EXPOSURE: Drug_ExposureView, + ModifierFieldConcepts.MEASUREMENT: MeasurementView, + ModifierFieldConcepts.OBSERVATION: ObservationView, + ModifierFieldConcepts.PROCEDURE_OCCURRENCE: Procedure_OccurrenceView, + } + + assert {field: targets[field] for field in expected} == expected + assert all(issubclass(view, ClinicalEventMixin) for view in expected.values()) + assert issubclass(EpisodeView, ModifierTargetMixin) + assert not issubclass(EpisodeView, ClinicalEventMixin) + assert all( + not issubclass(model, ModifierTargetMixin) + for model in ( + Condition_Occurrence, + Device_Exposure, + Drug_Exposure, + Measurement, + Observation, + Procedure_Occurrence, + ) + ) + + +def test_registered_event_views_have_distinct_field_concepts(): + views = ( + Condition_OccurrenceView, + Device_ExposureView, + Drug_ExposureView, + MeasurementView, + ObservationView, + Procedure_OccurrenceView, + ) + field_concepts = tuple(view.modifier_field_concept_id() for view in views) + + assert len(field_concepts) == len(set(field_concepts)) == 6 + + +def test_event_mixin_rejects_metadata_without_a_field_concept(): + class MissingFieldConcept(ClinicalEventMixin): + __event_id_col__ = "measurement_id" + __concept_id_col__ = "measurement_concept_id" + __start_date_col__ = "measurement_date" + + assert not MissingFieldConcept.has_complete_metadata() + with pytest.raises( + UnsupportedClinicalEventModelError, + match="no complete ClinicalEventMixin metadata", + ): + MissingFieldConcept.clinical_event_model_spec(Measurement) + + +def test_analytics_import_preserves_all_core_metadata_and_compiled_projections(): + code = textwrap.dedent( + """ + from sqlalchemy.dialects import postgresql, sqlite + from omop_alchemy.cdm.model import ( + Condition_Occurrence, + Device_Exposure, + Drug_Exposure, + Measurement, + Observation, + Procedure_Occurrence, + ) + from omop_alchemy.toolkit.core.events import canonical_event_projection + from omop_alchemy.cdm.model.clinical.event_metadata import clinical_event_model_spec + from omop_alchemy.cdm.model.structural import Episode_EventView + + models = ( + Condition_Occurrence, + Device_Exposure, + Drug_Exposure, + Measurement, + Observation, + Procedure_Occurrence, + ) + + def snapshot(): + return ( + tuple( + ( + model.__name__, + clinical_event_model_spec(model), + str(canonical_event_projection(model).compile(dialect=sqlite.dialect())), + str(canonical_event_projection(model).compile(dialect=postgresql.dialect())), + ) + for model in models + ), + tuple( + sorted( + (field, target.__name__) + for field, target in Episode_EventView.resolved_event_target_classes().items() + ) + ) + ) + + before = snapshot() + import omop_alchemy.toolkit.analytics.oncology + assert snapshot() == before + """ + ) + + subprocess.run([sys.executable, "-c", code], check=True) + + +@pytest.mark.parametrize( + "first_import", + [ + "omop_alchemy.cdm.base.event_metadata", + "omop_alchemy.cdm.model.clinical.event_metadata", + "omop_alchemy.toolkit.core.events.projections", + "omop_alchemy.toolkit.core.modifiers.metadata", + "omop_alchemy.toolkit.core.timeline.event_timeline", + ], +) +def test_metadata_import_order_keeps_metadata_in_cdm(first_import): + code = textwrap.dedent( + f""" + import importlib + import sys + importlib.import_module({first_import!r}) + from omop_alchemy.cdm.base.event_metadata import ( + ClinicalEventModelSpec, UnsupportedClinicalEventModelError, + ) + from omop_alchemy.cdm.model.clinical.event_metadata import clinical_event_model_spec + from omop_alchemy.cdm.model.clinical import Measurement + if {first_import!r}.startswith('omop_alchemy.cdm.'): + assert not any(name.startswith('omop_alchemy.toolkit') for name in sys.modules) + from omop_alchemy.toolkit.core import events + from omop_alchemy.toolkit.core.events import projections + assert not hasattr(events, 'ClinicalEventModelSpec') + assert not hasattr(events, 'UnsupportedClinicalEventModelError') + assert not hasattr(events, 'clinical_event_model_spec') + assert not hasattr(projections, 'ClinicalEventModelSpec') + assert not hasattr(projections, 'UnsupportedClinicalEventModelError') + assert not hasattr(projections, 'clinical_event_model_spec') + assert isinstance(clinical_event_model_spec(Measurement), ClinicalEventModelSpec) + """ + ) + subprocess.run([sys.executable, "-c", code], check=True) diff --git a/tests/test_event_timeline.py b/tests/test_event_timeline.py new file mode 100644 index 0000000..ca4ec57 --- /dev/null +++ b/tests/test_event_timeline.py @@ -0,0 +1,151 @@ +"""Behavioural coverage for the lightweight clinical event timeline.""" + +from __future__ import annotations + +from datetime import date, datetime +import json + +import sqlalchemy as sa +import pytest +from sqlalchemy.dialects import sqlite + +from omop_alchemy.cdm.base import ModifierFieldConcepts +from omop_alchemy.cdm.model.clinical import MeasurementView, ObservationView +from omop_alchemy.toolkit.core.events import ClinicalEventRow +from omop_alchemy.toolkit.core.timeline.event_timeline import EventMapping +from omop_alchemy.toolkit.core.timeline import ( + ClinicalEvent, + Condition_Event, + Drug_Exposure_Event, + Measurement_Event, + Observation_Event, + Person_Timeline, +) + + +def test_observation_event_implements_the_timeline_contract(): + event = Observation_Event( + observation_id=7, + person_id=101, + observation_concept_id=900_001, + observation_date=date(2026, 1, 20), + observation_datetime=datetime(2026, 1, 20, 9, 30), + observation_type_concept_id=32817, + value_as_string="family history", + ) + + assert isinstance(event, ClinicalEventRow) + assert event.event_id == 7 + assert event.event_source_table == "observation" + assert event.event_field_concept_id == ModifierFieldConcepts.OBSERVATION + assert event.event_concept_id == 900_001 + assert event.event_date == date(2026, 1, 20) + assert event.event_time.start == datetime(2026, 1, 20, 9, 30) + assert event.event_time.end is None + assert event.event_value().type == "string" + assert event.event_value().value == "family history" + + payload = json.loads(event.to_json()) + assert payload["event_id"] == 7 + assert payload["event_source_table"] == "observation" + assert payload["event_concept_id"] == 900_001 + assert "concept_id" not in payload + + +def test_observation_event_is_part_of_person_timeline_and_compiles(): + assert Observation_Event in Person_Timeline.EVENT_TABLES + + statement = sa.select(Observation_Event).where(Observation_Event.person_id == 101) + compiled = str(statement.compile(dialect=sqlite.dialect())) + + assert "observation" in compiled + assert "person_id" in compiled + + +def test_drug_exposure_quantity_is_a_numeric_timeline_value(): + event = Drug_Exposure_Event( + drug_exposure_id=8, + person_id=101, + drug_concept_id=900_002, + drug_exposure_start_date=date(2026, 1, 21), + drug_type_concept_id=32817, + quantity=12.5, + ) + + assert event.event_value().type == "numeric" + assert event.event_value().value == 12.5 + assert event.to_dict()["value"] == {"type": "numeric", "value": 12.5} + + +def test_measurement_event_only_declares_measurement_value_columns(): + assert Measurement_Event._mapping.value_fields == [ + "value_as_concept_id", + "value_as_number", + ] + + +def test_measurement_and_observation_views_expose_schema_backed_unit_context(): + measurement_relationships = MeasurementView.__mapper__.relationships + observation_relationships = ObservationView.__mapper__.relationships + + assert "unit_concept" in measurement_relationships + assert "unit_source_concept" in measurement_relationships + assert "unit_concept" in observation_relationships + assert "unit_source_concept" not in observation_relationships + + +def test_all_timeline_events_use_clinical_event_behaviour(): + assert Condition_Event.to_json is ClinicalEvent.to_json + assert Drug_Exposure_Event.to_json is ClinicalEvent.to_json + assert Observation_Event.to_json is ClinicalEvent.to_json + + +@pytest.mark.parametrize("model", [Measurement_Event, Observation_Event]) +@pytest.mark.parametrize("has_datetime", [False, True]) +def test_single_date_events_remain_points_with_inferred_metadata(model, has_datetime): + mapping = EventMapping.from_model(model) + fields = {mapping.start_date_field: date(2026, 9, 17)} + if has_datetime: + fields[mapping.start_datetime_field] = datetime(2026, 9, 17, 9, 30) + event = model(**fields) + assert mapping.end_date_field is None + assert mapping.end_datetime_field is None + assert event.event_time.kind == "point" + assert event.to_dict()["event_end"] is None + + +@pytest.mark.parametrize("model", [Condition_Event, Drug_Exposure_Event]) +@pytest.mark.parametrize("has_datetime", [False, True]) +def test_interval_events_infer_independent_endpoints(model, has_datetime): + mapping = EventMapping.from_model(model) + fields = { + mapping.start_date_field: date(2026, 9, 16), + mapping.end_date_field: date(2026, 9, 17), + } + expected_end = datetime(2026, 9, 17, 23, 59, 59, 999999) + if has_datetime: + fields[mapping.start_datetime_field] = datetime(2026, 9, 16, 9, 30) + fields[mapping.end_datetime_field] = expected_end = datetime( + 2026, 9, 17, 11, 30 + ) + event = model(**fields) + assert event.event_time.kind == "interval" + assert event.event_time.end == expected_end + assert event.to_dict()["event_end"] == expected_end.isoformat() + + +def test_explicit_endpoint_overrides_can_disable_inferred_intervals(): + mapping = EventMapping.from_model( + Condition_Event, + end_date_field=None, + end_datetime_field=None, + ) + assert mapping.end_date_field is None + assert mapping.end_datetime_field is None + overridden = EventMapping.from_model( + Condition_Event, + end_date_field="custom_end_date", + end_datetime_field="custom_end_datetime", + ) + assert overridden.end_date_field == "custom_end_date" + assert overridden.end_datetime_field == "custom_end_datetime" diff --git a/tests/test_load_vocab_postgres.py b/tests/test_load_vocab_postgres.py index 7e33727..df905a6 100644 --- a/tests/test_load_vocab_postgres.py +++ b/tests/test_load_vocab_postgres.py @@ -1,11 +1,9 @@ """ PostgreSQL integration tests for OMOP_Alchemy vocabulary loading. -These tests require a running PostgreSQL container. Start one with: - docker compose -f tests/docker-compose.yaml up -d - -Then run: - pytest -m postgres +These tests require a dedicated ``test_cdm_db`` PostgreSQL resource configured +with ``test_only = true``. Then run: + pytest -m requires_database """ from pathlib import Path diff --git a/tests/test_modifier_projections.py b/tests/test_modifier_projections.py new file mode 100644 index 0000000..4a6d215 --- /dev/null +++ b/tests/test_modifier_projections.py @@ -0,0 +1,566 @@ +"""Canonical modifier projection and target-resolution contracts.""" + +from __future__ import annotations + +import pytest +import sqlalchemy as sa +import sqlalchemy.orm as so +from datetime import date, datetime +from sqlalchemy.dialects import postgresql, sqlite + +from omop_alchemy.cdm.base import ( + ModifierFieldConcepts, + ModifierSourceMixin, + ClinicalEventMixin, +) +from orm_loader.helpers import Base +from omop_alchemy.cdm.model import ( + Condition_Occurrence, + Device_Exposure, + Drug_Exposure, + Measurement, + Observation, + Person, + Procedure_Occurrence, +) +from omop_alchemy.cdm.model.clinical import MeasurementView, ObservationView +from omop_alchemy.cdm.model.structural import Episode, Episode_EventView +from omop_alchemy.cdm.base.event_metadata import ClinicalEventModelSpec +from omop_alchemy.cdm.model.clinical.event_metadata import ( + CLINICAL_EVENT_TARGETS_BY_FIELD_CONCEPT_ID, + CLINICAL_EVENT_TARGETS_BY_TABLE, + MODIFIER_TARGETS_BY_TABLE, + STRUCTURAL_MODIFIER_TARGETS_BY_TABLE, + clinical_event_model_spec, + clinical_event_target_for_table, +) +from omop_alchemy.toolkit.core.modifiers.projections import _VALUE_COLUMN_TYPES +from omop_alchemy.toolkit.core.modifiers.contracts import ( + ModifierColumn, + ModifierRow, + ValuedModifierRow, +) +from omop_alchemy.toolkit.core.modifiers import ( + CDM_MODIFIER_SOURCE_MODELS, + MODIFIER_SOURCE_MODEL_SPECS_BY_TABLE, + MODIFIER_TARGET_SPECS_BY_TABLE, + UnsupportedModifierSourceModelError, + UnsupportedModifierTargetError, + canonical_modifier_projection, + canonical_modifier_target_projection, + canonical_modifier_union, + modifier_source_model_spec, + modifier_target_model_spec, + modifier_target_queries, +) + + +# The full projection shape, in UNION position order. Only the tests need the +# whole; production code asks for the obligation it actually cares about. +_ALL_MODIFIER_COLUMNS = ( + ModifierColumn.required_columns() + ModifierColumn.value_columns() +) + + +@pytest.mark.parametrize( + ("model", "source_table"), + [(Measurement, "measurement"), (Observation, "observation")], +) +def test_modifier_projection_has_one_stable_shape(model, source_table: str): + statement = canonical_modifier_projection(model) + + assert tuple(statement.selected_columns.keys()) == tuple( + map(str, _ALL_MODIFIER_COLUMNS) + ) + compiled = str( + statement.compile( + dialect=postgresql.dialect(), compile_kwargs={"literal_binds": True} + ) + ) + assert f"'{source_table}' AS modifier_source_table" in compiled + + +def test_modifier_union_preserves_shape_and_all_rows(): + statement = canonical_modifier_union(Measurement, Observation) + + assert tuple(statement.selected_columns.keys()) == tuple( + map(str, _ALL_MODIFIER_COLUMNS) + ) + assert "UNION ALL" in str(statement.compile(dialect=sqlite.dialect())) + + +def test_non_modifier_model_fails_at_query_construction(): + with pytest.raises( + UnsupportedModifierSourceModelError, match="ModifierSourceMixin" + ) as raised: + canonical_modifier_projection(Person) + + assert raised.value.model is Person + assert raised.value.reason == ( + "must declare the OMOP modifier link via ModifierSourceMixin" + ) + + +def test_unsupported_modifier_target_error_preserves_model_and_reason(): + with pytest.raises( + UnsupportedModifierTargetError, + match="is not a supported modifier target", + ) as raised: + modifier_target_model_spec(Person) + + assert raised.value.model is Person + assert raised.value.reason == "is not a supported modifier target" + + +def test_modifier_metadata_reuses_generic_model_interfaces(): + # The target link is no longer described by the spec at all; it is read off + # ModifierSourceMixin, which is asserted separately. + assert set(MODIFIER_SOURCE_MODEL_SPECS_BY_TABLE) == {"measurement", "observation"} + for table, spec in MODIFIER_SOURCE_MODEL_SPECS_BY_TABLE.items(): + # A source spec IS the clinical-event spec; the modifier-flavoured + # renaming happens on the projection's output labels, not here. + assert isinstance(spec, ClinicalEventModelSpec) + assert spec.event_source_table == table + + +def test_modifier_target_surface_is_the_clinical_and_structural_registries(): + assert set(MODIFIER_TARGETS_BY_TABLE) == { + *CLINICAL_EVENT_TARGETS_BY_TABLE, + *STRUCTURAL_MODIFIER_TARGETS_BY_TABLE, + } + assert not set(CLINICAL_EVENT_TARGETS_BY_TABLE) & set( + STRUCTURAL_MODIFIER_TARGETS_BY_TABLE + ) + assert set(STRUCTURAL_MODIFIER_TARGETS_BY_TABLE) == {"episode"} + + +def test_structural_groupers_never_enter_the_clinical_event_registry(): + """Episode groups clinical events, so it must never be resolvable as one. + + Admitting it would let an ``episode_event`` row resolve to an Episode, so a + grouper would appear inside its own ``EpisodeView.events`` list, and would + let canonical event projections double-count a grouper alongside the events + it groups. + """ + for table_name in STRUCTURAL_MODIFIER_TARGETS_BY_TABLE: + assert table_name not in CLINICAL_EVENT_TARGETS_BY_TABLE + assert clinical_event_target_for_table(table_name) is None + + structural_field_concepts = { + target.modifier_field_concept_id() + for target in STRUCTURAL_MODIFIER_TARGETS_BY_TABLE.values() + } + assert not structural_field_concepts & set( + CLINICAL_EVENT_TARGETS_BY_FIELD_CONCEPT_ID + ) + assert not structural_field_concepts & set( + Episode_EventView.resolved_event_target_classes() + ) + + +def test_canonical_column_vocabulary_is_stated_once(): + """The enum, its column groups, and the row protocols must agree. + + Each names the same columns for a different audience, and nothing in the + language keeps them in step, so the agreement is asserted here. + """ + assert not set(ModifierColumn.required_columns()) & set( + ModifierColumn.value_columns() + ), "a column cannot be both required and an optional value" + assert set(_ALL_MODIFIER_COLUMNS) == set(ModifierColumn), ( + "every ModifierColumn must be classified as required or value" + ) + assert len(_ALL_MODIFIER_COLUMNS) == len(ModifierColumn) + + assert tuple(ModifierRow.__annotations__) == tuple( + map(str, ModifierColumn.required_columns()) + ) + assert tuple(ValuedModifierRow.__annotations__) == tuple( + map(str, ModifierColumn.value_columns()) + ) + + # The projection casts each value position when a source lacks the column, + # so every value column needs a declared SQL type to fall back to. + assert set(_VALUE_COLUMN_TYPES) == set(ModifierColumn.value_columns()) + + +def test_modifier_sources_declare_the_link_through_the_source_mixin(): + """The common vocabulary must resolve to each table's own physical columns.""" + expected = { + Measurement: ("measurement_event_id", "meas_event_field_concept_id"), + Observation: ("observation_event_id", "obs_event_field_concept_id"), + } + for model, (event_column, field_column) in expected.items(): + assert issubclass(model, ModifierSourceMixin) + assert model.modifier_source_table() == model.__tablename__ + for hybrid, column_name in ( + (model.modifier_of_event_id, event_column), + (model.modifier_of_field_concept_id, field_column), + ): + # The hybrid renders as an annotated form of the mapped column, so + # compare semantically rather than by object identity. + rendered = hybrid.__clause_element__() + assert rendered.compare(model.__table__.c[column_name]) + assert rendered.table is model.__table__ + + +def _make_custom_source(**overrides): + """Build a mapped modifier source outside the CDM, on its own registry.""" + + class LocalBase(so.DeclarativeBase): + pass + + class CustomSource(LocalBase, ModifierSourceMixin, ClinicalEventMixin): + __tablename__ = "custom_source" + __event_id_col__ = "custom_source_id" + __concept_id_col__ = "custom_concept_id" + __start_date_col__ = "custom_date" + __end_date_col__ = "custom_date" + __type_concept_id_col__ = "custom_concept_id" + __modifier_event_id_col__ = overrides.get( + "__modifier_event_id_col__", "custom_event_id" + ) + __modifier_field_concept_id_col__ = "custom_event_field_concept_id" + + custom_source_id: so.Mapped[int] = so.mapped_column(primary_key=True) + person_id: so.Mapped[int] + custom_concept_id: so.Mapped[int] + custom_date: so.Mapped[date] + custom_datetime: so.Mapped[datetime | None] + custom_event_id: so.Mapped[int | None] + custom_event_field_concept_id: so.Mapped[int | None] + + @classmethod + def modifier_field_concept_id(cls) -> int: + return 999999 + + return CustomSource + + +def test_a_new_modifier_source_needs_no_change_to_the_toolkit(): + """Support is a property of the model, not a list held in this package.""" + custom = _make_custom_source() + + assert custom not in CDM_MODIFIER_SOURCE_MODELS + assert tuple(canonical_modifier_projection(custom).selected_columns.keys()) == ( + tuple(map(str, _ALL_MODIFIER_COLUMNS)) + ) + + spec = modifier_source_model_spec(custom) + assert spec.event_source_table == "custom_source" + assert spec.event_id_column == "custom_source_id" + assert custom.has_complete_metadata() + assert custom.clinical_event_model_spec() == spec + assert custom.modifier_target_table() not in MODIFIER_TARGETS_BY_TABLE + assert custom.modifier_field_concept_id() not in ( + Episode_EventView.resolved_event_target_classes() + ) + + union = canonical_modifier_union(Measurement, custom) + assert "UNION ALL" in str(union.compile(dialect=sqlite.dialect())) + + +def test_a_source_naming_a_missing_link_column_is_rejected(): + """The mixin supplies the hybrids; the subclass must name real columns.""" + custom = _make_custom_source(__modifier_event_id_col__="no_such_column") + + with pytest.raises( + UnsupportedModifierSourceModelError, match="names a missing column" + ): + modifier_source_model_spec(custom) + + +@pytest.mark.parametrize("source_model", [MeasurementView, ObservationView]) +def test_modifier_source_subclass_projects_its_own_event_metadata(source_model): + original = clinical_event_model_spec(source_model) + specialized = type( + f"{source_model.__name__}WithOverriddenEventMetadata", + (source_model,), + { + "__event_id_col__": "projected_id", + "projected_id": so.column_property( + getattr(source_model, original.event_id_column) + 100 + ), + "__concept_id_col__": "value_as_concept_id", + "__start_date_col__": "projected_date", + "projected_date": so.synonym(original.event_date_column), + "projected_datetime": so.synonym(original.event_datetime_column), + }, + ) + spec = modifier_source_model_spec(specialized) + assert spec == clinical_event_model_spec(specialized) + assert spec.event_date_column == "projected_date" + assert spec.event_datetime_column == "projected_datetime" + + engine = sa.create_engine("sqlite://") + Base.metadata.create_all(engine, tables=[source_model.__table__]) + with engine.begin() as connection: + connection.execute( + source_model.__table__.insert(), + { + original.event_id_column: 7, + "person_id": 101, + original.event_concept_id_column: 900_001, + original.event_date_column: date(2026, 1, 20), + source_model.__type_concept_id_col__: 32817, + "value_as_concept_id": 900_002, + }, + ) + row = ( + connection.execute(canonical_modifier_projection(specialized)) + .mappings() + .one() + ) + engine.dispose() + + assert row["modifier_id"] == 107 + assert row["modifier_concept_id"] == 900_002 + assert row["modifier_date"] == date(2026, 1, 20) + + +@pytest.mark.parametrize("source_model", [MeasurementView, ObservationView]) +@pytest.mark.parametrize( + "declaration", ["__event_id_col__", "__concept_id_col__", "__start_date_col__"] +) +def test_modifier_source_subclass_rejects_missing_declared_columns( + source_model, declaration +): + specialized = type( + f"{source_model.__name__}WithMissing{declaration.strip('_')}", + (source_model,), + {declaration: "no_such_column"}, + ) + with pytest.raises(UnsupportedModifierSourceModelError, match="no_such_column"): + canonical_modifier_projection(specialized) + + +def test_modifier_targets_derive_the_clinical_surface_and_extend_it_with_episode(): + assert set(MODIFIER_TARGET_SPECS_BY_TABLE) == { + *CLINICAL_EVENT_TARGETS_BY_TABLE, + "episode", + } + assert all( + MODIFIER_TARGET_SPECS_BY_TABLE[table].event_source_table == table + for table in MODIFIER_TARGET_SPECS_BY_TABLE + ) + + +def test_episode_is_a_modifier_target_without_becoming_a_clinical_event(): + spec = modifier_target_model_spec(Episode) + projection = canonical_modifier_target_projection(Episode) + + assert spec.event_field_concept_id == ModifierFieldConcepts.EPISODE + assert tuple(projection.selected_columns.keys()) == ( + "person_id", + "event_id", + "event_field_concept_id", + "event_source_table", + ) + + +@pytest.mark.parametrize( + "target", + [ + Condition_Occurrence, + Device_Exposure, + Drug_Exposure, + Measurement, + Observation, + Procedure_Occurrence, + Episode, + ], +) +def test_target_resolution_queries_compile_for_supported_models(target): + queries = modifier_target_queries( + Measurement, target, include_unmatched=True, diagnostics=True + ) + + assert queries.diagnostics is not None + for dialect in (sqlite.dialect(), postgresql.dialect()): + assert "resolved_target_event_id" in str( + queries.matches.compile(dialect=dialect) + ) + assert "diagnostic_code" in str(queries.diagnostics.compile(dialect=dialect)) + + +def _seeded_engine(conditions, measurements): + """Build a SQLite database holding the given condition and measurement rows.""" + engine = sa.create_engine("sqlite://") + Base.metadata.create_all( + engine, tables=[Condition_Occurrence.__table__, Measurement.__table__] + ) + with engine.begin() as connection: + connection.execute(Condition_Occurrence.__table__.insert(), conditions) + connection.execute(Measurement.__table__.insert(), measurements) + return engine + + +def _condition(condition_occurrence_id: int, person_id: int): + return dict( + condition_occurrence_id=condition_occurrence_id, + person_id=person_id, + condition_concept_id=4, + condition_start_date=date(2020, 1, 1), + condition_type_concept_id=1, + ) + + +def _modifier_of(measurement_id: int, person_id: int, target_id: int): + return dict( + measurement_id=measurement_id, + person_id=person_id, + measurement_concept_id=9, + measurement_date=date(2020, 6, 1), + measurement_type_concept_id=1, + measurement_event_id=target_id, + meas_event_field_concept_id=ModifierFieldConcepts.CONDITION_OCCURRENCE, + ) + + +def _diagnostic_codes(engine, target) -> set[tuple[int, str]]: + queries = modifier_target_queries(Measurement, target, diagnostics=True) + assert queries.diagnostics is not None + with engine.connect() as connection: + rows = connection.execute(queries.diagnostics).mappings().all() + return {(row["modifier_id"], row["diagnostic_code"]) for row in rows} + + +def test_a_whole_target_table_can_prove_a_dangling_modifier(): + """Absence from an entire table does mean the target row does not exist.""" + engine = _seeded_engine( + conditions=[_condition(1, 10)], + measurements=[_modifier_of(100, 10, 1), _modifier_of(101, 10, 99)], + ) + + assert _diagnostic_codes(engine, Condition_Occurrence) == { + (101, "missing_target_event") + } + + +def test_a_filtered_target_never_reports_a_missing_target_event(): + """A narrowed target set cannot distinguish a defect from its own filter. + + Both modifiers below point at conditions that genuinely exist. Reporting + the excluded one as missing would turn the caller's filter into a false + data-quality defect for downstream baseline counts. + """ + engine = _seeded_engine( + conditions=[_condition(1, 10), _condition(2, 10)], + measurements=[_modifier_of(100, 10, 1), _modifier_of(101, 10, 2)], + ) + narrowed = sa.select( + *canonical_modifier_target_projection(Condition_Occurrence).subquery().c + ).where(sa.column("event_id") == 2) + + assert _diagnostic_codes(engine, narrowed) == set() + + +def test_a_filtered_target_still_reports_person_mismatch(): + """A mismatch is observed on a row that is present, so it stays provable.""" + engine = _seeded_engine( + conditions=[_condition(1, 10)], + measurements=[_modifier_of(100, 11, 1)], + ) + narrowed = sa.select( + *canonical_modifier_target_projection(Condition_Occurrence).subquery().c + ).where(sa.column("event_id") == 1) + + assert _diagnostic_codes(engine, narrowed) == {(100, "person_mismatch")} + + +def test_target_validation_rejects_null_identity_and_cross_person_links(): + def modifier( + modifier_id: int, + person_id: int, + target_id: int | None, + target_field: int | None, + ): + return sa.select( + sa.literal(person_id).label("person_id"), + sa.literal(modifier_id).label("modifier_id"), + sa.literal("2025-01-01").label("modifier_date"), + sa.cast(sa.null(), sa.DateTime()).label("modifier_datetime"), + sa.literal(100).label("modifier_concept_id"), + sa.literal("measurement").label("modifier_source_table"), + sa.literal(target_id).label("target_event_id"), + sa.literal(target_field).label("target_field_concept_id"), + ) + + modifiers = sa.union_all( + modifier(1, 10, 7, ModifierFieldConcepts.CONDITION_OCCURRENCE), + modifier(2, 10, None, None), + modifier(3, 10, 8, ModifierFieldConcepts.CONDITION_OCCURRENCE), + modifier(4, 11, 7, ModifierFieldConcepts.CONDITION_OCCURRENCE), + ) + targets = sa.select( + sa.literal(10).label("person_id"), + sa.literal(7).label("event_id"), + sa.literal(ModifierFieldConcepts.CONDITION_OCCURRENCE).label( + "event_field_concept_id" + ), + sa.literal("condition_occurrence").label("event_source_table"), + ) + queries = modifier_target_queries(modifiers, targets, diagnostics=True) + + engine = sa.create_engine("sqlite://") + with engine.connect() as connection: + matches = connection.execute(queries.matches).mappings().all() + assert queries.diagnostics is not None + diagnostics = connection.execute(queries.diagnostics).mappings().all() + + assert [row["modifier_id"] for row in matches] == [1] + # Modifier 3 points outside the supplied selectable, which cannot prove the + # row is absent from the underlying table, so no code is asserted for it. + assert {(row["modifier_id"], row["diagnostic_code"]) for row in diagnostics} == { + (2, "missing_target_identity"), + (4, "person_mismatch"), + } + + +@pytest.mark.requires_database("test_cdm_db") +def test_postgresql_executes_modifier_target_validation_contracts(pg_session): + """PostgreSQL rejects incomplete and cross-person links before selection.""" + + def modifier( + modifier_id: int, + person_id: int, + target_id: int | None, + target_field: int | None, + ) -> sa.Select: + return sa.select( + sa.literal(person_id).label("person_id"), + sa.literal(modifier_id).label("modifier_id"), + sa.literal(date(2025, 1, 1)).label("modifier_date"), + sa.literal(datetime(2025, 1, 1, 9)).label("modifier_datetime"), + sa.literal(100).label("modifier_concept_id"), + sa.literal("measurement").label("modifier_source_table"), + sa.literal(target_id).label("target_event_id"), + sa.literal(target_field).label("target_field_concept_id"), + ) + + modifiers = sa.union_all( + modifier(1, 10, 7, ModifierFieldConcepts.CONDITION_OCCURRENCE), + modifier(2, 10, None, None), + modifier(3, 11, 7, ModifierFieldConcepts.CONDITION_OCCURRENCE), + ) + targets = sa.select( + sa.literal(10).label("person_id"), + sa.literal(7).label("event_id"), + sa.literal(ModifierFieldConcepts.CONDITION_OCCURRENCE).label( + "event_field_concept_id" + ), + sa.literal("condition_occurrence").label("event_source_table"), + ) + queries = modifier_target_queries(modifiers, targets, diagnostics=True) + + assert [ + row["modifier_id"] for row in pg_session.execute(queries.matches).mappings() + ] == [1] + assert queries.diagnostics is not None + assert { + (row["modifier_id"], row["diagnostic_code"]) + for row in pg_session.execute(queries.diagnostics).mappings() + } == { + (2, "missing_target_identity"), + (3, "person_mismatch"), + } diff --git a/tests/test_modifier_selection.py b/tests/test_modifier_selection.py new file mode 100644 index 0000000..486b06a --- /dev/null +++ b/tests/test_modifier_selection.py @@ -0,0 +1,334 @@ +"""Deterministic modifier selection and oncology stage preferences.""" + +from __future__ import annotations + +from datetime import date, datetime + +import pytest +import sqlalchemy as sa + +from omop_alchemy.toolkit.analytics.oncology import ( + DEFAULT_STAGE_SELECTION, + GROUP_STAGE_CONCEPTS, + M_STAGE_CONCEPTS, + METASTATIC_DISEASE_CONCEPTS, + N_STAGE_CONCEPTS, + StageBasis, + StageSelectionSpec, + T_STAGE_CONCEPTS, + TUMOR_GRADE_CONCEPTS, + laterality_modifier_concept_id, + preferred_stage_select, + tumor_size_modifier_concept_id, +) +from omop_alchemy.toolkit.core.modifiers import ( + ModifierSelectionPolicy, + selected_modifier_select, +) + + +def _stage_source(*, reverse: bool = False) -> sa.CompoundSelect: + rows = ( + # Clinical is chronologically first, but pathological wins by default. + (1, 10, date(2025, 1, 1), datetime(2025, 1, 1, 9), 100, "cT2"), + (1, 11, date(2025, 2, 1), datetime(2025, 2, 1, 9), 101, "pT2"), + (1, 12, date(2025, 3, 1), datetime(2025, 3, 1, 9), 102, "pT3"), + ) + branches = [] + if reverse: + rows = tuple(reversed(rows)) + for ( + person_id, + modifier_id, + modifier_date, + modifier_datetime, + concept_id, + code, + ) in rows: + branches.append( + sa.select( + sa.literal(person_id).label("person_id"), + sa.literal(modifier_id).label("modifier_id"), + sa.literal(modifier_date).label("modifier_date"), + sa.literal(modifier_datetime).label("modifier_datetime"), + sa.literal(concept_id).label("modifier_concept_id"), + sa.literal("measurement").label("modifier_source_table"), + sa.literal(500).label("target_event_id"), + sa.literal(1147127).label("target_field_concept_id"), + sa.literal(code).label("modifier_concept_code"), + ) + ) + return sa.union_all(*branches) + + +@pytest.mark.parametrize( + ("spec", "expected_id"), + [ + (DEFAULT_STAGE_SELECTION, 11), + (StageSelectionSpec.clinical_first(), 10), + (StageSelectionSpec.chronological_only(), 10), + ( + StageSelectionSpec( + temporal_policy=ModifierSelectionPolicy.latest, + ), + 12, + ), + ], +) +def test_stage_selection_default_and_overrides(spec, expected_id: int): + engine = sa.create_engine("sqlite://") + with engine.connect() as connection: + selected = ( + connection.execute( + preferred_stage_select( + _stage_source(), + spec=spec, + concept_code_column="modifier_concept_code", + ) + ) + .mappings() + .one() + ) + + assert selected["modifier_id"] == expected_id + + +def test_stage_selection_spec_is_immutable_and_validates_basis_permutation(): + with pytest.raises(AttributeError): + DEFAULT_STAGE_SELECTION.basis_priority = (StageBasis.clinical,) # type: ignore[misc] + + with pytest.raises(ValueError, match="each StageBasis exactly once"): + StageSelectionSpec(basis_priority=(StageBasis.pathological,)) + + +def test_basis_selection_requires_an_explicit_concept_code_column(): + with pytest.raises( + ValueError, + match="concept_code_column is required when basis ranking is enabled", + ): + preferred_stage_select(_stage_source()) + + +def test_chronological_policy_does_not_require_a_concept_code_column(): + source = _stage_source().subquery() + without_code = sa.select( + *(column for column in source.c if column.key != "modifier_concept_code") + ) + + preferred_stage_select(without_code, spec=StageSelectionSpec.chronological_only()) + + +def test_same_basis_tie_is_stable_when_input_order_reverses(): + engine = sa.create_engine("sqlite://") + selected_ids = [] + with engine.connect() as connection: + for reverse in (False, True): + source = _stage_source(reverse=reverse).subquery() + tied = sa.select(source).where(source.c.modifier_id.in_((11, 12))) + # Make both pathological candidates a true temporal tie. + tied_source = tied.subquery() + normalized = sa.select( + *( + sa.literal(date(2025, 2, 1)).label(column.key) + if column.key == "modifier_date" + else sa.literal(datetime(2025, 2, 1, 9)).label(column.key) + if column.key == "modifier_datetime" + else column + for column in tied_source.c + ) + ) + selected_ids.append( + connection.execute( + preferred_stage_select( + normalized, + concept_code_column="modifier_concept_code", + ) + ) + .mappings() + .one()["modifier_id"] + ) + + assert selected_ids == [11, 11] + + +def test_same_numeric_target_id_in_two_target_tables_forms_two_partitions(): + source = _stage_source().subquery() + condition = sa.select(source).where(source.c.modifier_id == 10) + procedure = sa.select( + *( + sa.literal(1147082).label(column.key) + if column.key == "target_field_concept_id" + else column + for column in source.c + ) + ).where(source.c.modifier_id == 11) + + engine = sa.create_engine("sqlite://") + with engine.connect() as connection: + rows = ( + connection.execute( + selected_modifier_select(sa.union_all(condition, procedure)) + ) + .mappings() + .all() + ) + + assert {(row["target_field_concept_id"], row["modifier_id"]) for row in rows} == { + (1147127, 10), + (1147082, 11), + } + + +def test_condition_modifier_specs_delegate_to_governed_semantics(): + pytest.importorskip("omop_semantics") + from omop_semantics.runtime.default_valuesets import runtime + + group_pairs = ( + (T_STAGE_CONCEPTS, runtime.staging.t_stage_concepts), + (N_STAGE_CONCEPTS, runtime.staging.n_stage_concepts), + (M_STAGE_CONCEPTS, runtime.staging.m_stage_concepts), + (GROUP_STAGE_CONCEPTS, runtime.staging.group_stage_concepts), + ( + METASTATIC_DISEASE_CONCEPTS, + runtime.condition_modifiers.metastatic_disease_concepts, + ), + ) + for spec, unit in group_pairs: + assert set(spec.parent_ids()) == set(unit.parent_ids) + assert set(TUMOR_GRADE_CONCEPTS.exact_ids()) == set( + runtime.condition_modifiers.tumor_grade.exact_ids + ) + assert ( + laterality_modifier_concept_id() + == runtime.condition_modifiers.condition_modifier_values.laterality + ) + assert ( + tumor_size_modifier_concept_id() + == runtime.condition_modifiers.numeric_condition_modifiers.tumor_size + ) + + +@pytest.mark.requires_database("test_cdm_db") +def test_postgresql_executes_modifier_selection_and_stage_policy_contracts(pg_session): + """Execute collision, stable-tie, and stage overrides on PostgreSQL.""" + expected_by_spec = ( + (DEFAULT_STAGE_SELECTION, 11), + (StageSelectionSpec.clinical_first(), 10), + (StageSelectionSpec.chronological_only(), 10), + ( + StageSelectionSpec(temporal_policy=ModifierSelectionPolicy.latest), + 12, + ), + ) + for spec, expected_id in expected_by_spec: + selected = ( + pg_session.execute( + preferred_stage_select( + _stage_source(), + spec=spec, + concept_code_column="modifier_concept_code", + ) + ) + .mappings() + .one() + ) + assert selected["modifier_id"] == expected_id + + stage_source = _stage_source().subquery() + unclassified = sa.select( + *( + sa.literal("uT2").label(column.key) + if column.key == "modifier_concept_code" + else column + for column in stage_source.c + ) + ).where(stage_source.c.modifier_id == 10) + assert ( + pg_session.execute( + preferred_stage_select( + unclassified, + concept_code_column="modifier_concept_code", + ) + ) + .mappings() + .one()["modifier_id"] + == 10 + ) + + # A true same-basis temporal tie must use the canonical source identity, + # independently of branch order in the input UNION ALL. + selected_ids = [] + for reverse in (False, True): + source = _stage_source(reverse=reverse).subquery() + pathological = sa.select(source).where(source.c.modifier_id.in_((11, 12))) + tied = pathological.subquery() + normalized = sa.select( + *( + sa.literal(date(2025, 2, 1)).label(column.key) + if column.key == "modifier_date" + else sa.literal(datetime(2025, 2, 1, 9)).label(column.key) + if column.key == "modifier_datetime" + else column + for column in tied.c + ) + ) + selected_ids.append( + pg_session.execute( + preferred_stage_select( + normalized, + concept_code_column="modifier_concept_code", + ) + ) + .mappings() + .one()["modifier_id"] + ) + assert selected_ids == [11, 11] + + # A shared numeric target ID remains two partitions when the OMOP Field + # concept differs; each target therefore retains its own winner. + source = _stage_source().subquery() + condition = sa.select(source).where(source.c.modifier_id == 10) + procedure = sa.select( + *( + sa.literal(1147082).label(column.key) + if column.key == "target_field_concept_id" + else column + for column in source.c + ) + ).where(source.c.modifier_id == 11) + selected = pg_session.execute( + selected_modifier_select(sa.union_all(condition, procedure)) + ).mappings().all() + assert { + (row["target_field_concept_id"], row["modifier_id"]) for row in selected + } == { + (1147127, 10), + (1147082, 11), + } + + # Source-table scope keeps equal native IDs distinct. They also target two + # different OMOP tables that happen to use the same numeric event ID. + def modifier(source_table: str, target_field: int) -> sa.Select: + return sa.select( + sa.literal(1).label("person_id"), + sa.literal(7).label("modifier_id"), + sa.literal(date(2025, 1, 1)).label("modifier_date"), + sa.literal(datetime(2025, 1, 1, 9)).label("modifier_datetime"), + sa.literal(100).label("modifier_concept_id"), + sa.literal(source_table).label("modifier_source_table"), + sa.literal(500).label("target_event_id"), + sa.literal(target_field).label("target_field_concept_id"), + ) + + equal_native_ids = pg_session.execute( + selected_modifier_select( + sa.union_all( + modifier("measurement", 1147127), + modifier("observation", 1147082), + ) + ) + ).mappings().all() + assert { + (row["modifier_source_table"], row["modifier_id"]) + for row in equal_native_ids + } == {("measurement", 7), ("observation", 7)} diff --git a/tests/test_query_builder_contracts.py b/tests/test_query_builder_contracts.py new file mode 100644 index 0000000..ecb0647 --- /dev/null +++ b/tests/test_query_builder_contracts.py @@ -0,0 +1,326 @@ +"""Query contracts and reusable counterexamples for SQL builders.""" + +from __future__ import annotations + +from datetime import date, timedelta + +import pytest +import sqlalchemy as sa +from sqlalchemy.dialects import postgresql, sqlite + +from omop_alchemy.cdm.base import ModifierFieldConcepts +from omop_alchemy.toolkit.core.concepts import ( + RuntimeConceptSetSpec, +) +from omop_alchemy.toolkit.core.events import ( + ClinicalEventColumn, + ClinicalEventIdentity, + ClinicalEventRow, +) +from omop_alchemy.toolkit.core.timeline import Measurement_Event +from omop_alchemy.toolkit.episodes.derivation import ( + EpisodeAttachmentIdentity, + EpisodeAttachmentPolicy, + EpisodeWindowSpec, + ObservationSelectionPolicy, + ObservationSelectionSpec, + TemporalRankingSpec, + TemporalSelectionPolicy, + TemporalSidePreference, +) +from tests.fixtures.query_contract_cases import ( + BOUNDARY_EVENTS, + COLLIDING_EVENTS, + CONSTRUCTS_EXACT_180_DAY_EPISODE, + CONSTRUCTS_FUTURE_VISIT_EPISODES, + CROSS_PERSON_LINK, + DIRECTIONAL_PREFERENCE_EPISODES, + OBSERVATION_ANCHOR_DATE, + OVERLAPPING_EPISODES, + REPEATED_OBSERVATIONS, + VALID_EXPLICIT_LINK, + COLLIDING_VALID_LINK, + OUT_OF_SCOPE_LINK, +) + + +def test_canonical_event_shape_has_unique_stable_names(): + all_columns = ( + ClinicalEventColumn.required_columns() + ClinicalEventColumn.optional_columns() + ) + + assert len(all_columns) == len(set(all_columns)) + assert tuple(str(column) for column in ClinicalEventColumn.required_columns()) == ( + "person_id", + "event_id", + "event_date", + "event_datetime", + "event_concept_id", + "event_field_concept_id", + "event_source_table", + ) + + +def test_source_table_is_required_to_distinguish_colliding_event_ids(): + identities = {case.identity for case in COLLIDING_EVENTS} + + assert {case.identity.event_id for case in COLLIDING_EVENTS} == {7} + assert len(identities) == 3 + assert ClinicalEventIdentity("measurement", 7) != ClinicalEventIdentity( + "procedure_occurrence", 7 + ) + + +def test_timeline_event_implements_the_shared_core_event_contract(): + event = Measurement_Event( + measurement_id=7, + person_id=101, + measurement_concept_id=900_001, + measurement_date=date(2026, 1, 20), + measurement_datetime=None, + ) + + assert isinstance(event, ClinicalEventRow) + assert event.event_id == 7 + assert event.event_source_table == "measurement" + assert event.event_field_concept_id == ModifierFieldConcepts.MEASUREMENT + assert event.event_concept_id == 900_001 + assert event.event_date == date(2026, 1, 20) + assert event.event_datetime is None + + +def test_attachment_identity_keeps_the_event_source_and_episode(): + event = ClinicalEventIdentity("procedure_occurrence", 7) + + first = EpisodeAttachmentIdentity.from_event(event, episode_id=1001) + second = EpisodeAttachmentIdentity.from_event(event, episode_id=1002) + + assert first.event == event + assert first != second + + +@pytest.mark.parametrize( + "identity_type", [ClinicalEventIdentity, EpisodeAttachmentIdentity] +) +def test_event_source_table_cannot_be_empty(identity_type): + args = ("", 7) if identity_type is ClinicalEventIdentity else ("", 7, 1001) + with pytest.raises(ValueError, match="event_source_table"): + identity_type(*args) + + +@pytest.mark.parametrize( + ("policy", "uses_fallback", "permits_fanout"), + [ + (EpisodeAttachmentPolicy.explicit_only, False, False), + (EpisodeAttachmentPolicy.explicit_first_ranked, True, False), + (EpisodeAttachmentPolicy.explicit_first_all_in_window, True, True), + ], +) +def test_attachment_policies_state_precedence_and_cardinality( + policy: EpisodeAttachmentPolicy, + uses_fallback: bool, + permits_fanout: bool, +): + assert policy.uses_fallback is uses_fallback + assert policy.permits_fallback_fanout is permits_fanout + assert policy.requires_fallback_ranking is ( + policy is EpisodeAttachmentPolicy.explicit_first_ranked + ) + + +def test_counterexample_links_cover_valid_out_of_scope_and_person_cases(): + event_by_identity = {case.identity: case for case in COLLIDING_EVENTS} + episode_by_id = {case.episode_id: case for case in OVERLAPPING_EPISODES} + + valid_event = event_by_identity[VALID_EXPLICIT_LINK.event] + assert ( + valid_event.event_field_concept_id + == VALID_EXPLICIT_LINK.episode_event_field_concept_id + ) + assert ( + valid_event.person_id == episode_by_id[VALID_EXPLICIT_LINK.episode_id].person_id + ) + + colliding_event = event_by_identity[COLLIDING_VALID_LINK.event] + assert ( + colliding_event.event_field_concept_id + == COLLIDING_VALID_LINK.episode_event_field_concept_id + ) + assert all( + event.event_field_concept_id != OUT_OF_SCOPE_LINK.episode_event_field_concept_id + for event in COLLIDING_EVENTS + ) + + cross_person_event = event_by_identity[CROSS_PERSON_LINK.event] + assert ( + cross_person_event.person_id + != episode_by_id[CROSS_PERSON_LINK.episode_id].person_id + ) + + +def test_nearest_temporal_contract_uses_absolute_distance_and_stable_id(): + spec = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + ) + event_date = date(2026, 1, 20) + ranked = sorted( + OVERLAPPING_EPISODES[:2], + key=lambda episode: ( + abs((event_date - episode.start_date).days), + episode.episode_id, + ), + ) + + assert spec.uses_absolute_distance + assert [episode.episode_id for episode in ranked] == [1001, 1002] + + +def test_started_episode_preference_can_override_absolute_nearest(): + anchor = date(2026, 1, 20) + neutral = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + ) + started_first = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + side_preference=TemporalSidePreference.on_or_before_anchor, + ) + + absolute = sorted( + DIRECTIONAL_PREFERENCE_EPISODES, + key=lambda episode: ( + abs((episode.start_date - anchor).days), + episode.episode_id, + ), + ) + directional = sorted( + DIRECTIONAL_PREFERENCE_EPISODES, + key=lambda episode: ( + episode.start_date > anchor, + abs((episode.start_date - anchor).days), + episode.episode_id, + ), + ) + + assert not neutral.has_side_preference + assert started_first.has_side_preference + assert absolute[0].episode_id == 1003 + assert directional[0].episode_id == 1001 + + +def test_constructs_visit_fixture_exposes_signed_future_distance_pitfall(): + event_date = date(2026, 1, 20) + signed = sorted( + CONSTRUCTS_FUTURE_VISIT_EPISODES, + key=lambda episode: (event_date - episode.start_date).days, + ) + absolute = sorted( + CONSTRUCTS_FUTURE_VISIT_EPISODES, + key=lambda episode: abs((event_date - episode.start_date).days), + ) + + assert signed[0].episode_id == 4002 + assert absolute[0].episode_id == 4001 + + +def test_constructs_visit_fixture_records_strict_180_day_boundary(): + event_date = date(2026, 1, 20) + distance = abs((event_date - CONSTRUCTS_EXACT_180_DAY_EPISODE.start_date).days) + + assert distance == 180 + assert not distance < 180 + + +def test_boundary_fixture_makes_closed_window_expectations_visible(): + spec = EpisodeWindowSpec() + episode = OVERLAPPING_EPISODES[0] + lower = episode.start_date - timedelta(days=90) + upper = episode.end_date + assert upper is not None + included = [ + event.identity.event_id + for event in BOUNDARY_EVENTS + if lower <= event.event_date <= upper + ] + + assert spec.include_lower_bound + assert spec.include_upper_bound + assert included == [8, 10] + + +def test_observation_as_of_contract_excludes_future_and_breaks_ties_by_id(): + spec = ObservationSelectionSpec( + policy=ObservationSelectionPolicy.latest_on_or_before_anchor + ) + candidates = [ + row + for row in REPEATED_OBSERVATIONS + if row.observation_date <= OBSERVATION_ANCHOR_DATE + ] + selected = sorted( + candidates, + key=lambda row: (-row.observation_date.toordinal(), row.observation_id), + )[0] + + assert spec.requires_anchor + assert selected.observation_id == 22 + + +def test_runtime_concept_set_is_normalised_without_database_access(): + spec = RuntimeConceptSetSpec( + include_ancestor_ids=(300, 100, 300), + include_exact_ids=(900,), + exclude_ancestor_ids=(400,), + exclude_exact_ids=(901, 901), + require_standard=True, + include_classification=False, + ) + + assert spec.include_ancestor_ids == (100, 300) + assert spec.exclude_exact_ids == (901,) + assert spec.has_inclusions + assert spec.require_standard + + +def test_runtime_concept_set_does_not_invent_concept_id_validity_policy(): + spec = RuntimeConceptSetSpec(include_exact_ids=(0, -1, 0)) + + assert spec.include_exact_ids == (-1, 0) + + +def _projection_contract_select() -> sa.Select: + """Minimal selectable proving the canonical labels compile on supported dialects.""" + event = sa.table( + "research_event", + sa.column("person_id", sa.Integer), + sa.column("event_id", sa.Integer), + sa.column("event_date", sa.Date), + sa.column("event_concept_id", sa.Integer), + ) + return sa.select( + event.c.person_id.label(ClinicalEventColumn.person_id), + event.c.event_id.label(ClinicalEventColumn.event_id), + event.c.event_date.label(ClinicalEventColumn.event_date), + sa.cast(sa.null(), sa.DateTime).label(ClinicalEventColumn.event_datetime), + event.c.event_concept_id.label(ClinicalEventColumn.event_concept_id), + sa.literal(ModifierFieldConcepts.MEASUREMENT).label( + ClinicalEventColumn.event_field_concept_id + ), + sa.literal("measurement").label(ClinicalEventColumn.event_source_table), + ) + + +@pytest.mark.parametrize("dialect", [sqlite.dialect(), postgresql.dialect()]) +def test_required_projection_contract_compiles_without_execution(dialect): + statement = _projection_contract_select() + compiled = str( + statement.compile(dialect=dialect, compile_kwargs={"literal_binds": True}) + ) + + assert tuple(statement.selected_columns.keys()) == tuple( + str(column) for column in ClinicalEventColumn.required_columns() + ) + assert "event_field_concept_id" in compiled + assert "event_source_table" in compiled diff --git a/tests/test_reference_context.py b/tests/test_reference_context.py new file mode 100644 index 0000000..764d766 --- /dev/null +++ b/tests/test_reference_context.py @@ -0,0 +1,116 @@ +"""Reference joins derive mapper identity while preserving deferred loading.""" + +import pytest +import sqlalchemy as sa +import sqlalchemy.orm as so + +from omop_alchemy.cdm.base import ReferenceContext + + +@pytest.fixture(params=[None, "identifier"]) +def reference_models(request): + class LocalBase(so.DeclarativeBase): + pass + + # Declare the source before its target to exercise deferred resolution. + class Source(LocalBase): + __tablename__ = "source" + id: so.Mapped[int] = so.mapped_column(primary_key=True) + target_identifier: so.Mapped[str] + reference = ReferenceContext._reference_relationship( + target="Target", + local_fk="target_identifier", + remote_pk=request.param, + ) + + class Target(LocalBase): + __tablename__ = "target" + identifier: so.Mapped[str] = so.mapped_column("physical_key", primary_key=True) + alternative: so.Mapped[str] + + try: + yield LocalBase, Source, Target + finally: + LocalBase.registry.dispose() + + +def test_reference_join_uses_mapped_attribute_and_batches_loading(reference_models): + base, source, target = reference_models + relationship = sa.inspect(source).relationships.reference + assert relationship.primaryjoin.compare( + source.target_identifier == target.identifier + ) + assert relationship.viewonly + assert relationship.lazy == "selectin" + assert not relationship.uselist + + engine = sa.create_engine("sqlite://") + base.metadata.create_all(engine) + with so.Session(engine) as session: + session.add_all( + [ + target(identifier="pk-1", alternative="other-1"), + source(id=1, target_identifier="pk-1"), + source(id=2, target_identifier="pk-1"), + ] + ) + session.commit() + statements = [] + sa.event.listen( + engine, "before_cursor_execute", lambda *args: statements.append(args[2]) + ) + try: + with so.Session(engine) as session: + rows = session.scalars(sa.select(source).order_by(source.id)).all() + assert [row.reference.identifier for row in rows] == ["pk-1", "pk-1"] + assert len(statements) == 2 + finally: + engine.dispose() + + +@pytest.mark.parametrize( + "reference_models", ["alternative", "physical_key"], indirect=True +) +def test_wrong_legacy_reference_key_is_rejected(reference_models): + base, _, _ = reference_models + with pytest.raises(ValueError, match="primary key is 'identifier', not"): + base.registry.configure() + + +def test_missing_reference_target_is_rejected(reference_models): + base, _, _ = reference_models + + class MissingSource(base): + __tablename__ = "missing_source" + id: so.Mapped[int] = so.mapped_column(primary_key=True) + target_id: so.Mapped[int] + reference = ReferenceContext._reference_relationship( + target="Missing", + local_fk="target_id", + ) + + with pytest.raises( + ValueError, match="reference target 'Missing' is missing or ambiguous" + ): + base.registry.configure() + + +def test_composite_reference_key_is_rejected(reference_models): + base, _, _ = reference_models + + class CompositeSource(base): + __tablename__ = "composite_source" + id: so.Mapped[int] = so.mapped_column(primary_key=True) + target_id: so.Mapped[int] + reference = ReferenceContext._reference_relationship( + target="Composite", + local_fk="target_id", + ) + + class Composite(base): + __tablename__ = "composite" + id: so.Mapped[int] = so.mapped_column(primary_key=True) + other_id: so.Mapped[int] = so.mapped_column(primary_key=True) + + with pytest.raises(ValueError, match="exactly one mapped primary key"): + base.registry.configure() diff --git a/tests/test_runtime_concept_queries.py b/tests/test_runtime_concept_queries.py new file mode 100644 index 0000000..60b0ff7 --- /dev/null +++ b/tests/test_runtime_concept_queries.py @@ -0,0 +1,198 @@ +"""Database-side runtime concept hierarchy predicates.""" + +from __future__ import annotations + +from datetime import date + +import sqlalchemy as sa +from sqlalchemy.dialects import postgresql, sqlite + +from omop_alchemy.cdm.model.vocabulary import Concept, Concept_Ancestor +from omop_alchemy.toolkit.core.concepts import ( + RuntimeConceptSetSpec, + descendant_concept_select, + runtime_concept_predicate, +) + + +def _add_runtime_hierarchy(session) -> None: + concepts = ( + (990_000, "runtime root", "S"), + (990_001, "included standard", "S"), + (990_002, "included classification", "C"), + (990_003, "excluded standard", "S"), + (990_010, "runtime exclusion root", "S"), + ) + session.add_all( + Concept( + concept_id=concept_id, + concept_name=name, + domain_id="Condition", + vocabulary_id="SNOMED", + concept_class_id="Clinical Finding", + standard_concept=standardness, + concept_code=str(concept_id), + valid_start_date=date(1970, 1, 1), + valid_end_date=date(2099, 12, 31), + ) + for concept_id, name, standardness in concepts + ) + session.flush() + session.add_all( + ( + Concept_Ancestor( + ancestor_concept_id=990_000, + descendant_concept_id=990_001, + min_levels_of_separation=1, + max_levels_of_separation=1, + ), + Concept_Ancestor( + ancestor_concept_id=990_000, + descendant_concept_id=990_002, + min_levels_of_separation=1, + max_levels_of_separation=1, + ), + Concept_Ancestor( + ancestor_concept_id=990_000, + descendant_concept_id=990_003, + min_levels_of_separation=1, + max_levels_of_separation=1, + ), + Concept_Ancestor( + ancestor_concept_id=990_010, + descendant_concept_id=990_003, + min_levels_of_separation=1, + max_levels_of_separation=1, + ), + ) + ) + session.flush() + + +def test_runtime_hierarchy_applies_union_then_exclusion_in_database(session): + _add_runtime_hierarchy(session) + spec = RuntimeConceptSetSpec( + include_ancestor_ids=(990_000,), + include_exact_ids=(8507,), + exclude_ancestor_ids=(990_010,), + exclude_exact_ids=(990_001,), + ) + + statement = sa.select(Concept.concept_id).where( + runtime_concept_predicate(Concept.concept_id, spec) + ) + assert session.scalars(statement.order_by(Concept.concept_id)).all() == [ + 8507, + 990_002, + ] + + +def test_runtime_hierarchy_reuses_standardness_policy(session): + _add_runtime_hierarchy(session) + standard_only = RuntimeConceptSetSpec( + include_ancestor_ids=(990_000,), + require_standard=True, + include_classification=False, + ) + with_classification = RuntimeConceptSetSpec( + include_ancestor_ids=(990_000,), + require_standard=True, + include_classification=True, + ) + + standard_only_statement = sa.select(Concept.concept_id).where( + runtime_concept_predicate(Concept.concept_id, standard_only) + ) + with_classification_statement = sa.select(Concept.concept_id).where( + runtime_concept_predicate(Concept.concept_id, with_classification) + ) + + assert session.scalars(standard_only_statement).all() == [990_001, 990_003] + assert set(session.scalars(with_classification_statement)) == { + 990_001, + 990_002, + 990_003, + } + + +def test_runtime_hierarchy_with_no_inclusions_is_always_false(session): + statement = sa.select(Concept.concept_id).where( + runtime_concept_predicate(Concept.concept_id, RuntimeConceptSetSpec()) + ) + + assert session.scalars(statement).all() == [] + + +def test_runtime_hierarchy_compiles_to_concept_ancestor_subqueries(): + statement = sa.select(Concept.concept_id).where( + runtime_concept_predicate( + Concept.concept_id, + RuntimeConceptSetSpec( + include_ancestor_ids=(100,), + exclude_ancestor_ids=(400,), + require_standard=True, + ), + ) + ) + + for dialect in (sqlite.dialect(), postgresql.dialect()): + compiled = str(statement.compile(dialect=dialect)) + assert "concept_ancestor" in compiled + assert "standard_concept" in compiled + + +def test_descendant_select_returns_overlapping_descendant_once(session): + _add_runtime_hierarchy(session) + + descendants = session.scalars(descendant_concept_select((990_000, 990_010))).all() + + assert descendants.count(990_003) == 1 + + +def test_exact_runtime_id_does_not_depend_on_a_concept_row(session): + configured_id = -1 + statement = sa.select(sa.literal(configured_id)).where( + runtime_concept_predicate( + sa.literal(configured_id), + RuntimeConceptSetSpec(include_exact_ids=(configured_id,)), + ) + ) + + assert session.scalar(statement) == configured_id + + +def test_exclusion_wins_over_an_exact_inclusion(session): + _add_runtime_hierarchy(session) + spec = RuntimeConceptSetSpec( + include_exact_ids=(990_003,), + exclude_ancestor_ids=(990_010,), + ) + statement = sa.select(Concept.concept_id).where( + runtime_concept_predicate(Concept.concept_id, spec) + ) + + assert session.scalars(statement).all() == [] + + +def test_exact_inclusions_are_not_removed_by_descendant_standardness_policy(session): + _add_runtime_hierarchy(session) + standard_only = RuntimeConceptSetSpec( + include_exact_ids=(990_002,), + require_standard=True, + include_classification=False, + ) + with_classification = RuntimeConceptSetSpec( + include_exact_ids=(990_002,), + require_standard=True, + include_classification=True, + ) + + standard_only_statement = sa.select(Concept.concept_id).where( + runtime_concept_predicate(Concept.concept_id, standard_only) + ) + with_classification_statement = sa.select(Concept.concept_id).where( + runtime_concept_predicate(Concept.concept_id, with_classification) + ) + + assert session.scalars(standard_only_statement).all() == [990_002] + assert session.scalars(with_classification_statement).all() == [990_002] diff --git a/tests/test_temporal_queries.py b/tests/test_temporal_queries.py new file mode 100644 index 0000000..f6d48ba --- /dev/null +++ b/tests/test_temporal_queries.py @@ -0,0 +1,247 @@ +"""Portable temporal and repeated-observation SQL expressions.""" + +from __future__ import annotations + +from datetime import date + +import pytest +import sqlalchemy as sa +from sqlalchemy.dialects import mysql, postgresql, sqlite +from sqlalchemy.exc import UnsupportedCompilationError + +from omop_alchemy.toolkit.episodes.derivation import ( + ObservationSelectionPolicy, + ObservationSelectionSpec, + EpisodeWindowSpec, + TemporalRankingSpec, + TemporalSelectionPolicy, + TemporalSidePreference, + absolute_day_delta, + bounded_temporal_predicate, + episode_window_bounds, + ranked_observation_select, + signed_day_delta, + temporal_order_expressions, +) + + +def _temporal_candidates() -> sa.CTE: + return sa.union_all( + sa.select( + sa.literal(1001).label("episode_id"), + sa.literal(date(2026, 1, 15)).label("start_date"), + ), + sa.select( + sa.literal(1003).label("episode_id"), + sa.literal(date(2026, 1, 21)).label("start_date"), + ), + ).cte("temporal_candidates") + + +@pytest.mark.parametrize("dialect", [sqlite.dialect(), postgresql.dialect()]) +def test_day_delta_compiles_for_supported_dialects(dialect): + statement = sa.select( + signed_day_delta(sa.literal(date(2026, 1, 21)), sa.literal(date(2026, 1, 20))), + absolute_day_delta( + sa.literal(date(2026, 1, 15)), + sa.literal(date(2026, 1, 20)), + ), + ) + + compiled = str(statement.compile(dialect=dialect)) + if dialect.name == "sqlite": + assert "julianday" in compiled + else: + assert "CAST" in compiled + + +def test_day_delta_rejects_an_unsupported_dialect(): + statement = sa.select( + signed_day_delta(sa.literal(date(2026, 1, 21)), sa.literal(date(2026, 1, 20))) + ) + + with pytest.raises(UnsupportedCompilationError): + statement.compile(dialect=mysql.dialect()) + + +def test_episode_window_rejects_an_unsupported_dialect(): + lower, upper = episode_window_bounds( + sa.literal(date(2026, 1, 15)), + sa.literal(None, type_=sa.Date()), + ) + + with pytest.raises(UnsupportedCompilationError): + sa.select(lower, upper).compile(dialect=mysql.dialect()) + + +def test_day_delta_executes_as_signed_calendar_days(session): + values = session.execute( + sa.select( + signed_day_delta( + sa.literal(date(2026, 1, 21)), + sa.literal(date(2026, 1, 20)), + ), + signed_day_delta( + sa.literal(date(2026, 1, 15)), + sa.literal(date(2026, 1, 20)), + ), + ) + ).one() + + assert tuple(values) == (1, -5) + + +def test_side_preference_is_applied_before_absolute_distance(session): + candidates = _temporal_candidates() + anchor = sa.literal(date(2026, 1, 20)) + neutral = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + ) + started_first = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + side_preference=TemporalSidePreference.on_or_before_anchor, + ) + + def first_id(spec: TemporalRankingSpec) -> int: + statement = sa.select(candidates.c.episode_id).order_by( + *temporal_order_expressions( + candidates.c.start_date, + anchor, + candidates.c.episode_id, + spec, + ) + ) + value = session.scalar(statement.limit(1)) + assert value is not None + return value + + assert first_id(neutral) == 1003 + assert first_id(started_first) == 1001 + + +def test_episode_window_bounds_are_finite_and_boundary_policy_is_explicit(session): + start = sa.literal(date(2026, 1, 15)) + end = sa.literal(None, type_=sa.Date()) + lower, upper = episode_window_bounds( + start, + end, + window=EpisodeWindowSpec(days_prior=90, open_end_fallback_days=30), + ) + values = session.execute(sa.select(lower, upper)).one() + + assert tuple(values) == (date(2025, 10, 17), date(2026, 2, 14)) + closed = bounded_temporal_predicate( + sa.literal(date(2025, 10, 17)), + lower, + upper, + ) + open_lower = bounded_temporal_predicate( + sa.literal(date(2025, 10, 17)), + lower, + upper, + include_lower_bound=False, + ) + assert session.scalar(sa.select(closed)) is True + assert session.scalar(sa.select(open_lower)) is False + + +def _observation_source() -> sa.CTE: + rows = ( + (21, date(2026, 1, 1)), + (22, date(2026, 1, 20)), + (23, date(2026, 1, 20)), + (24, date(2026, 1, 21)), + ) + return sa.union_all( + *( + sa.select( + sa.literal(101).label("person_id"), + sa.literal(900_001).label("observation_concept_id"), + sa.literal(observation_id).label("observation_id"), + sa.literal(observation_date).label("observation_date"), + ) + for observation_id, observation_date in rows + ) + ).cte("observations") + + +def test_as_of_observation_selection_filters_before_ranking(session): + source = _observation_source() + spec = ObservationSelectionSpec( + policy=ObservationSelectionPolicy.latest_on_or_before_anchor + ) + ranked = ranked_observation_select( + source, + spec, + anchor_date=sa.literal(date(2026, 1, 20)), + ).subquery() + selected = session.scalar( + sa.select(ranked.c.observation_id).where(ranked.c.observation_rank == 1) + ) + + assert selected == 22 + + +def test_as_of_observation_selection_can_exclude_the_anchor_date(session): + source = _observation_source() + spec = ObservationSelectionSpec( + policy=ObservationSelectionPolicy.latest_on_or_before_anchor, + include_anchor_date=False, + ) + ranked = ranked_observation_select( + source, + spec, + anchor_date=sa.literal(date(2026, 1, 20)), + ).subquery() + + assert ( + session.scalar( + sa.select(ranked.c.observation_id).where(ranked.c.observation_rank == 1) + ) + == 21 + ) + + +def test_as_of_observation_selection_requires_an_anchor(): + with pytest.raises(ValueError, match="requires anchor_date"): + ranked_observation_select( + _observation_source(), + ObservationSelectionSpec( + policy=ObservationSelectionPolicy.latest_on_or_before_anchor + ), + ) + + +@pytest.mark.requires_database("test_cdm_db") +def test_postgresql_executes_boundary_and_side_preference_contracts(pg_session): + start = sa.literal(date(2026, 1, 15)) + end = sa.literal(date(2026, 2, 5)) + lower, upper = episode_window_bounds(start, end) + boundary = bounded_temporal_predicate( + sa.literal(date(2025, 10, 17)), + lower, + upper, + ) + assert pg_session.scalar(sa.select(boundary)) is True + + candidates = _temporal_candidates() + ranking = TemporalRankingSpec( + policy=TemporalSelectionPolicy.nearest, + stable_id_column="episode_id", + side_preference=TemporalSidePreference.on_or_before_anchor, + ) + selected = pg_session.scalar( + sa.select(candidates.c.episode_id) + .order_by( + *temporal_order_expressions( + candidates.c.start_date, + sa.literal(date(2026, 1, 20)), + candidates.c.episode_id, + ranking, + ) + ) + .limit(1) + ) + assert selected == 1001 diff --git a/uv.lock b/uv.lock index ca0977a..ef9443e 100644 --- a/uv.lock +++ b/uv.lock @@ -3,10 +3,10 @@ revision = 3 requires-python = ">=3.12" resolution-markers = [ "python_full_version >= '3.15' and sys_platform == 'win32'", - "python_full_version == '3.14.*' and sys_platform == 'win32'", "python_full_version >= '3.15' and sys_platform == 'emscripten'", - "python_full_version == '3.14.*' and sys_platform == 'emscripten'", "python_full_version >= '3.15' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.14.*' and sys_platform == 'win32'", + "python_full_version == '3.14.*' and sys_platform == 'emscripten'", "python_full_version == '3.14.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", "python_full_version < '3.14' and sys_platform == 'win32'", "python_full_version < '3.14' and sys_platform == 'emscripten'", @@ -46,15 +46,6 @@ version = "4.9.3" source = { registry = "https://pypi.org/simple" } sdist = { url = "https://files.pythonhosted.org/packages/3e/38/7859ff46355f76f8d19459005ca000b6e7012f2f1ca597746cbcd1fbfe5e/antlr4-python3-runtime-4.9.3.tar.gz", hash = "sha256:f224469b4168294902bb1efa80a8bf7855f24c99aef99cbefc1bcd3cce77881b", size = 117034, upload-time = "2021-11-06T17:52:23.524Z" } -[[package]] -name = "appnope" -version = "0.1.4" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/35/5d/752690df9ef5b76e169e68d6a129fa6d08a7100ca7f754c89495db3c6019/appnope-0.1.4.tar.gz", hash = "sha256:1de3860566df9caf38f01f86f65e0e13e379af54f9e4bee1e66b48f2efffd1ee", size = 4170, upload-time = "2024-02-06T09:43:11.258Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/81/29/5ecc3a15d5a33e31b26c11426c45c501e439cb865d0bff96315d86443b78/appnope-0.1.4-py2.py3-none-any.whl", hash = "sha256:502575ee11cd7a28c0205f379b525beefebab9d161b7c964670864014ed7213c", size = 4321, upload-time = "2024-02-06T09:43:09.663Z" }, -] - [[package]] name = "arrow" version = "1.4.0" @@ -177,91 +168,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/59/8c/57e832b7af6d7c5abe66eb3fbe3a3a32f4d11ea23a1aa7131371035be991/certifi-2026.5.20-py3-none-any.whl", hash = "sha256:3c52e209ba0a4ad7aebe60436a4ab349c39e1e602e8c134221e546902ad25897", size = 134134, upload-time = "2026-05-20T11:46:48.578Z" }, ] -[[package]] -name = "cffi" -version = "2.1.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "pycparser", marker = "implementation_name != 'PyPy'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/9e/ef/008a1939e372c06329a3fce4279c02f328488f3526744906eeec3da7ad5f/cffi-2.1.1.tar.gz", hash = "sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be", size = 530807, upload-time = "2026-08-03T21:21:18.939Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/10/69/43965eccfdead3b9220015fd1320e117be8c6ed01a62ffab76eeb752f5d5/cffi-2.1.1-cp312-cp312-macosx_10_15_x86_64.whl", hash = "sha256:c8c69575568085ba0b1b10c0249d779a214aea6f6522e949a0fc9fb0fcb449d0", size = 184821, upload-time = "2026-08-03T21:19:44.887Z" }, - { url = "https://files.pythonhosted.org/packages/54/7d/16e5a096677b5e313ca80cd5e5170efa3ea44624a82bb111925522da64b1/cffi-2.1.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f81b3b8f3d4e343550fa4baa0e479bba9f2d29ce9c2e9b51d1ce1718d7442fcf", size = 184719, upload-time = "2026-08-03T21:19:46.129Z" }, - { url = "https://files.pythonhosted.org/packages/56/e6/8941622732edec876dd17d0453dce07317ae96db34f2ec1436c9d3785986/cffi-2.1.1-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:811bd1e21d32de12efca32393a0ab3f5133b54fce9bd44b8bd77ab07da14bf6a", size = 214799, upload-time = "2026-08-03T21:19:47.218Z" }, - { url = "https://files.pythonhosted.org/packages/44/de/f98430906df1545ffde0d543dd124a7a439bc2cd32b36b9c53f805df7333/cffi-2.1.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:68e62fe11f30d5ca8289242866f0a5291402d8529ca2178ab8afc5c9694ae890", size = 222389, upload-time = "2026-08-03T21:19:48.331Z" }, - { url = "https://files.pythonhosted.org/packages/6a/5b/717f1526b9957b34456313c31645c5b82b8fb5c3fe9e4752999be7128bfc/cffi-2.1.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:4a7c934f7360e8cd64fe9efadcbd10c7c6364f531e432b9a4bf5ccbc9e0e8b50", size = 210249, upload-time = "2026-08-03T21:19:49.543Z" }, - { url = "https://files.pythonhosted.org/packages/64/b3/f8aa4f3e34986c7e4ec45072d1b1b9dd295b6b18007b45518d79726dd725/cffi-2.1.1-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:3143d81e29e1e20a9ce10901ec369012947876596f75a222235965f2b7ae832e", size = 208775, upload-time = "2026-08-03T21:19:50.918Z" }, - { url = "https://files.pythonhosted.org/packages/b1/db/dceb9dd5b231e1da801793f8acc9f3c52a7e1afe40bb1aae37e02b0faad5/cffi-2.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c1453022f490d2459a11819d83ad1d586e9ff65a12ac3e705ffebd46d3685dcf", size = 221822, upload-time = "2026-08-03T21:19:52.054Z" }, - { url = "https://files.pythonhosted.org/packages/a0/d2/6cd24ae3be000a634109c247d1475d62e5616d0dc78c82770942ec384248/cffi-2.1.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:208f941bb9d18e768138677f0a6d2ce01f590df56043dda1df1535ac57c88517", size = 225232, upload-time = "2026-08-03T21:19:53.109Z" }, - { url = "https://files.pythonhosted.org/packages/cb/52/3fa190537004dd7f0ab860a6dc7c0175b8667f68d1e618a46f5498d30250/cffi-2.1.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:210019b6c7cf07f081b4c54635c8cf744377001350e29cc0f81c4377b4797735", size = 223597, upload-time = "2026-08-03T21:19:54.515Z" }, - { url = "https://files.pythonhosted.org/packages/80/fb/0bb75b7039588c074b37ae99f40d9bfddf990ecb2fbc346ebccd2e56b9be/cffi-2.1.1-cp312-cp312-win32.whl", hash = "sha256:046bfc24911b37851ee1b51aab8bffe713d89c68c6a057b09484ce9fd5f69b4e", size = 175292, upload-time = "2026-08-03T21:19:55.566Z" }, - { url = "https://files.pythonhosted.org/packages/d9/79/615cc094e2fb508cade7de88d3b4f6c4ec2bab695c97bce9153dc65aadf5/cffi-2.1.1-cp312-cp312-win_amd64.whl", hash = "sha256:f53e442b08449d42821fa4a4fba000095af9f62742a500f978a9f557ec44339a", size = 185919, upload-time = "2026-08-03T21:19:56.89Z" }, - { url = "https://files.pythonhosted.org/packages/70/c6/d0ea84713fe46b243a436a18fcd47d639732747e21635c8a27191b06dc30/cffi-2.1.1-cp312-cp312-win_arm64.whl", hash = "sha256:7bde5e4cc5c10140859842b9d383af292b22639a4dffb725314baf45968cef80", size = 180093, upload-time = "2026-08-03T21:19:58.155Z" }, - { url = "https://files.pythonhosted.org/packages/9d/f4/035513d4117049066b4779dc3b7c0c0fdad175fa13731c9f4003f1cd1478/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:b5bdfd1c873d4e093aabc0ca84c4ca6dbc4f752afb5c86f146d9742580c9da2e", size = 194248, upload-time = "2026-08-03T21:19:59.399Z" }, - { url = "https://files.pythonhosted.org/packages/76/af/2aeb4dbb5fc41a04161ae9ff1518de7cec08e164f44a8ce6a4cf7fd2cd1d/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:31348097ff5bbe827ccc41795d4dd099d9f0625e7def00ee653c137a490c2a6c", size = 196908, upload-time = "2026-08-03T21:20:00.746Z" }, - { url = "https://files.pythonhosted.org/packages/a7/46/2e5fdde8555706dd98139a910ca11be02809f3f605ce956f655d0214e100/cffi-2.1.1-cp313-cp313-macosx_10_15_x86_64.whl", hash = "sha256:9d2055050ea716bd38b7f7f1579c275386646b4894c155a3e2f3cd62ed41b7c6", size = 184805, upload-time = "2026-08-03T21:20:02.02Z" }, - { url = "https://files.pythonhosted.org/packages/55/41/4c7042f317b9217502988f0873af87e16ad606dc20f84e546e3e6ce9764c/cffi-2.1.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:19ee6127ee34de7d83ce3d371ebc5ed91addbdcc39f9ab15ce4eb35a4e534971", size = 184764, upload-time = "2026-08-03T21:20:03.141Z" }, - { url = "https://files.pythonhosted.org/packages/43/1f/1c3d90d91811c8f86ced9ed637956c54bfe5b79ca98fe976d7f8c8979f6b/cffi-2.1.1-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:6a8dddef476fab96d066d578fc88526767b836ab5ab21754e1d5bf3879c31c7c", size = 214722, upload-time = "2026-08-03T21:20:04.377Z" }, - { url = "https://files.pythonhosted.org/packages/37/6f/3b5ce4c3b2192d250f04908f2bfd91ef34552ec8f7716a5d4abdb8d67bb2/cffi-2.1.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f16c709686a78c727bbbf059f92b0bf41c6fc60deec706d2dc19f529175a6125", size = 222369, upload-time = "2026-08-03T21:20:05.544Z" }, - { url = "https://files.pythonhosted.org/packages/02/10/4b3c75dde3d9663c9e02ba05c2668b954f671d4bbe346413ca8c696b295a/cffi-2.1.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:fcd22650c908d7b7da162bbfaab594a1227a15d1643a98c68b122ac642fa2264", size = 210175, upload-time = "2026-08-03T21:20:06.75Z" }, - { url = "https://files.pythonhosted.org/packages/df/62/14f74b9543e605d17701dc797b815958b8bb70b7624ce1b832ddad48ed6c/cffi-2.1.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:aa9511c62d14da7aacc9b4bf51f3f697a621e83b2d6919008243c3aad168eea3", size = 208670, upload-time = "2026-08-03T21:20:08.04Z" }, - { url = "https://files.pythonhosted.org/packages/95/95/86342356ff5953b3fb06f7ef7c5bee212d45e770abc7218d451b9148313c/cffi-2.1.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a931079504ecc49efed7744c476a5c343a92fabf66dec2db95edb1b2fdc770e2", size = 221824, upload-time = "2026-08-03T21:20:09.274Z" }, - { url = "https://files.pythonhosted.org/packages/eb/ff/7b3429ff53aafe931ed8a5fc69f481bbef7ba6de87ddcbb63d08f483f613/cffi-2.1.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a2d7755bef5a12ed488f4ef1f1b69ee9191d7396083b755a5d2295f6edb4768b", size = 225148, upload-time = "2026-08-03T21:20:10.7Z" }, - { url = "https://files.pythonhosted.org/packages/34/34/a95870b9221e09cf4f2ce3178b1a210abdfe63a1bd357da940418d7b8d15/cffi-2.1.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:e0bcb7e0f677f543555d2adff3bf19c05f66cdb4796e5ff602442ab2fe3c4ef7", size = 223564, upload-time = "2026-08-03T21:20:12.165Z" }, - { url = "https://files.pythonhosted.org/packages/70/ea/839b50531021a647fb5e929f72cf97bc1ff702b5472166164b5b6e76b851/cffi-2.1.1-cp313-cp313-win32.whl", hash = "sha256:334644fbac4eff73d985a17a91226df55d0f394160c4cfb880e084c8f7161cac", size = 175263, upload-time = "2026-08-03T21:20:13.559Z" }, - { url = "https://files.pythonhosted.org/packages/60/a6/8b149b2c3f2e11aaa1618ef64500b45f50f22c57a977a4dff1aff1f91042/cffi-2.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:1aa5645c30469b09530c4ebca77ebf8f17618293c58f8549cb1a543a50236e7d", size = 185688, upload-time = "2026-08-03T21:20:14.69Z" }, - { url = "https://files.pythonhosted.org/packages/01/9a/11f687cb39d6a3504060d5242f04f48c735afb4d3d533958a20594890cb2/cffi-2.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:63bbfd5ded17c4840ac07cd8f1c21ba9d9708141f840b324f422f41b207e3973", size = 180078, upload-time = "2026-08-03T21:20:15.917Z" }, - { url = "https://files.pythonhosted.org/packages/d3/7b/d6bbf82b8b96e7391438898c42f5bd96dd02030fd5b64937d248220003e2/cffi-2.1.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:7dbb61fe3a7699468030f71bbe5f8a0e326a151daa91beb11a6fc1f980c55e1c", size = 194064, upload-time = "2026-08-03T21:20:17.148Z" }, - { url = "https://files.pythonhosted.org/packages/94/e6/bcc91b283be94735e268487a054004f0aa19947b6348fa367db53230abc8/cffi-2.1.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:f24fb43132a4c6b4cb4eb029492919b2db645be6808d738f244fd146c03c32cb", size = 196720, upload-time = "2026-08-03T21:20:18.268Z" }, - { url = "https://files.pythonhosted.org/packages/d9/99/c4b0c17cacdc9c3b8f280026286a9826d6a208c0f047591a3c3ce99b91fd/cffi-2.1.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d28630f5854ab07ab1fd4aba756de52326c82e6be15d414b12793f1975048b54", size = 184964, upload-time = "2026-08-03T21:20:19.708Z" }, - { url = "https://files.pythonhosted.org/packages/b3/a9/9db617d05d7367c1ad0ab00b3aa6e6f9281edd689b4ee9ea0e5a84e89c97/cffi-2.1.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:661c298b4821edebead0c91edd2b00374d67ad7c5a1f7a91d4442633b79d6a72", size = 184962, upload-time = "2026-08-03T21:20:20.833Z" }, - { url = "https://files.pythonhosted.org/packages/67/b8/b42132ca113dc567d37684437b46ca1dafc885902b02a110a02d5b511857/cffi-2.1.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:58acb8ab8e295e6c5ea12f888cbb13cf21511ef2a3303a23f4325c29d17fe5c1", size = 222328, upload-time = "2026-08-03T21:20:22.118Z" }, - { url = "https://files.pythonhosted.org/packages/80/10/c5c0cbf0a657aecf59ef511409734230bf556f05a0d6c9eed7aa5c0a0166/cffi-2.1.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:456a61fa52d579ebf9df2e9552ead5129855dbaff6c1e5a9b1bc408809bdc062", size = 209985, upload-time = "2026-08-03T21:20:23.401Z" }, - { url = "https://files.pythonhosted.org/packages/d5/6c/bfa0b87b03b9238148beca990292843c9396ba069b54496596594173de7b/cffi-2.1.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a4f00aa42f75d6e4595e8866e748cc1705adc0cddfeb2ca86d0d03993d63ba03", size = 208530, upload-time = "2026-08-03T21:20:24.628Z" }, - { url = "https://files.pythonhosted.org/packages/e9/02/4e7d553a7ac4b4238b38b3c1b80d486e9d4436f8d2acbf87a0997fe3f402/cffi-2.1.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b0431303acaea1089ad4b3e9ce4e6518193def1118d4073ca848635ee4ea2e96", size = 221525, upload-time = "2026-08-03T21:20:25.758Z" }, - { url = "https://files.pythonhosted.org/packages/82/1d/a4aaf9babd75acb4d5f223bff71533bee748dd770a382619a798960ee9ba/cffi-2.1.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:64faea20f4e2613363a1a9b9c7dd73058f3ecd00133a511e72ad7c511658f527", size = 225053, upload-time = "2026-08-03T21:20:26.985Z" }, - { url = "https://files.pythonhosted.org/packages/81/10/5dc0e7bdd18e22107054288283380fc97a06ae3f1656a106908d666a3c88/cffi-2.1.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5c58fe613dc5e5336357eff555824a314d8e43282600435c8d1cb6a7a2fedd13", size = 223213, upload-time = "2026-08-03T21:20:28.277Z" }, - { url = "https://files.pythonhosted.org/packages/0b/e9/d0061c364cde06ee43168a0d076ac1da512cbc380d44767b844ba34fe2b6/cffi-2.1.1-cp314-cp314-win32.whl", hash = "sha256:1a18a57b58cfb21fc28d72e876acf10eaed67a1ed96226f92af4df681d571c4c", size = 177682, upload-time = "2026-08-03T21:20:44.288Z" }, - { url = "https://files.pythonhosted.org/packages/a7/06/1c3e01e3ba14c39f6d10bfbac52753b7e22259e38088e5cfe1d704918690/cffi-2.1.1-cp314-cp314-win_amd64.whl", hash = "sha256:3222ba5d678f80a030e6afbcc33dc1ae5cb45facabb61cee2c7016b8432fde48", size = 187949, upload-time = "2026-08-03T21:20:45.623Z" }, - { url = "https://files.pythonhosted.org/packages/87/5b/da4e39efe18eeb89cf580ea9cfc66b6a7c3eadb808fc0cc1d3a295cb5a5d/cffi-2.1.1-cp314-cp314-win_arm64.whl", hash = "sha256:ab36d55f9ed2d067327667c2fea18dda018eb628dd6347aa01dda6cf1f5d3836", size = 182947, upload-time = "2026-08-03T21:20:46.955Z" }, - { url = "https://files.pythonhosted.org/packages/23/59/40338bf421c5accea1d45158170c87006ef1cd371b05c077e76476949728/cffi-2.1.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:7750c6449dff7864bb9bb27ddfb0267756189201a3afc911d82b3caacd70dfc3", size = 188504, upload-time = "2026-08-03T21:20:29.495Z" }, - { url = "https://files.pythonhosted.org/packages/7d/47/5ecf1023850036e674c77ec4de86182d309ae344e39e7cba984b7df5d647/cffi-2.1.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0beceaabe56af686895136a2de78db54ecd8e4046b236b8fd6d6cb61389e9bf2", size = 188259, upload-time = "2026-08-03T21:20:31.291Z" }, - { url = "https://files.pythonhosted.org/packages/2a/9c/92934c3bea9f785b23eba304538c0b4d37a2a96d2431eb3a1bc87a11aa19/cffi-2.1.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:49cbc70e6542d4ccccb936558d1064a8012541e78f821f955cff24e357776c94", size = 223864, upload-time = "2026-08-03T21:20:32.571Z" }, - { url = "https://files.pythonhosted.org/packages/4d/45/ba4c93527bc38616a8bd36488acb69a2212d60486794f0c1f318949bbb76/cffi-2.1.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:e2d65b31f36619cda3999b78b2aa9632e76b78448e7a56fc4240824200e7c4fc", size = 211538, upload-time = "2026-08-03T21:20:33.808Z" }, - { url = "https://files.pythonhosted.org/packages/80/e9/b6ef565e452acb932fb0cb5443f44a78efbd1233e566f02b5a83855e9115/cffi-2.1.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:28907ab9bfb6aa13184cfc17c6b8e1023c5ab6fd7076d8c20a35e59fe04f8f29", size = 210688, upload-time = "2026-08-03T21:20:34.974Z" }, - { url = "https://files.pythonhosted.org/packages/9a/95/eff5f0cee78d2eabc7eebffec40d3fc1876b5f3c95582e018bb4b99601f2/cffi-2.1.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:51b31d1c98274844cfd7838ce00bfc27c7423a4dc00fc0772fc3331c2cc90676", size = 223803, upload-time = "2026-08-03T21:20:36.564Z" }, - { url = "https://files.pythonhosted.org/packages/fa/01/579d39fb8bef00a335a23d83757b44feb24cd6345a2c451b64cb67b9c362/cffi-2.1.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:5e7cecbaadb83884793e05828cee59b210b24583b9c7425d0ba6a754fe22eb4e", size = 226763, upload-time = "2026-08-03T21:20:37.816Z" }, - { url = "https://files.pythonhosted.org/packages/8d/b0/0b44f47c60b01b57b6e2bbd92343f13a85a1d93bc46ccf6e47e244acd99c/cffi-2.1.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:25792eac27877609e7bb06d42ff88278a6624fff2ba9bbb523c09616b117e80f", size = 225688, upload-time = "2026-08-03T21:20:38.959Z" }, - { url = "https://files.pythonhosted.org/packages/eb/d2/3b7176cb570a1d3e27faf67b72f591af508036e0d8b2be2ef9af9e8c84bb/cffi-2.1.1-cp314-cp314t-win32.whl", hash = "sha256:8ef53b2de9bcb9197d31854256575d59dbac0cba72ac627bb291ef5eceb74be4", size = 182868, upload-time = "2026-08-03T21:20:40.388Z" }, - { url = "https://files.pythonhosted.org/packages/56/78/31f00c1bcd97c9bbf55f1bfdf5bc809a5de8887473e90bb9960dca825e80/cffi-2.1.1-cp314-cp314t-win_amd64.whl", hash = "sha256:616f097f2fe415bc92a247f02e11f634e1f9e9a83d327e3c915c15089c87869e", size = 194104, upload-time = "2026-08-03T21:20:41.725Z" }, - { url = "https://files.pythonhosted.org/packages/7b/1b/58496f2ed0a35de575250c02a43ab3cc2c04d494a88fed31c1cabc0fd176/cffi-2.1.1-cp314-cp314t-win_arm64.whl", hash = "sha256:ad2c86c495b899d862ea0f4b42891b8713a3bd45dd4105c7fd51c2a72f39f3a5", size = 186402, upload-time = "2026-08-03T21:20:43.042Z" }, - { url = "https://files.pythonhosted.org/packages/c1/8f/9ebe220eab48a093d1a5a5e339ab0dc7316eef3bb04d63c42f0251b61f50/cffi-2.1.1-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:dddad92b554513a31f272570678ba307fb9f618f05e3d4a5eacafff9eae03e1d", size = 194043, upload-time = "2026-08-03T21:20:48.179Z" }, - { url = "https://files.pythonhosted.org/packages/ff/69/844bad3ece306c4782c2ecb93597035b6690d48704b803914c199da1e8b3/cffi-2.1.1-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:da0e573f9f97159390c89d9f1a9e41908b66d408cc5b58d08cf3847d844c531b", size = 196737, upload-time = "2026-08-03T21:20:49.457Z" }, - { url = "https://files.pythonhosted.org/packages/1b/8a/af668013284634733f02d683458a0728739c7d6ddb5e14cb0c20832266fe/cffi-2.1.1-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:fb92203a88b3d3053034db775110081c49d28be6551923805e039924093761e4", size = 184933, upload-time = "2026-08-03T21:20:50.639Z" }, - { url = "https://files.pythonhosted.org/packages/0c/75/2f5207ff6d1a613133b23a5203cc0c2a628313b5eb3974d7956ae3c57950/cffi-2.1.1-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:2ae64be792b8966f2c69538199728b290e34726562896df1e5dc8ffd8d8188e8", size = 185002, upload-time = "2026-08-03T21:20:52.173Z" }, - { url = "https://files.pythonhosted.org/packages/e2/31/9e1313b0a6e30e91b3b3d3fff51ae99c857c07738e3afcce1f7334e1b7ab/cffi-2.1.1-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:507a24c282e0f42f8ed737cf048572cbf580468da5555764a8331735e9c736b6", size = 222271, upload-time = "2026-08-03T21:20:53.462Z" }, - { url = "https://files.pythonhosted.org/packages/50/e3/f6234a833e6e08c7007003074723c406559eecf9b48dfc97471e5a8eb7a0/cffi-2.1.1-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:246fa40ce8645a614ff682e0b70f37134e460eaf93a775e0cbe3cca585a67a80", size = 209919, upload-time = "2026-08-03T21:20:54.783Z" }, - { url = "https://files.pythonhosted.org/packages/0d/fc/5f74e293fced6edb51af3a46c4ccf6c23c9943774ecb375ddbd522c76add/cffi-2.1.1-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:471cee653ae88de62096552e6d24ccb4a5adb8c8c9f10b5054d0122c15bf2779", size = 208529, upload-time = "2026-08-03T21:20:56.066Z" }, - { url = "https://files.pythonhosted.org/packages/44/16/29e6d01b388bef055ecd6ca8244b3f4d336bd09e92d5d892187b9601084e/cffi-2.1.1-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:aeae0e330c9f6acd681f647d46cefd30c29f93e3392882e792e82080c9691399", size = 221630, upload-time = "2026-08-03T21:20:57.336Z" }, - { url = "https://files.pythonhosted.org/packages/a4/18/fa7f1f6857d5eb88a4ca99ffcbfb7c387a287ccc154c64a73e86314745d7/cffi-2.1.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:42a494cee34437f05546455144f2b5d9ac09b1face62bcfce597d2e521066688", size = 225134, upload-time = "2026-08-03T21:20:58.675Z" }, - { url = "https://files.pythonhosted.org/packages/e0/9f/e8e3dfa04a1b4c241f8c91faacad872b4d4efd051d49764ad4e2fd4b9fea/cffi-2.1.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:cc572dace3f60ef98d7b12ff411d20f5362feb31a0439eab0085bbfd349982d7", size = 223197, upload-time = "2026-08-03T21:20:59.968Z" }, - { url = "https://files.pythonhosted.org/packages/f8/7e/8debeb04f1ab9fe2a6963964cd6f1aaf7192627b83926586a6a4e089c9fa/cffi-2.1.1-cp315-cp315-win32.whl", hash = "sha256:4f42141fc14250de6dde5ee7ea4432be017252d91f19c5ad043c084cea629cac", size = 177683, upload-time = "2026-08-03T21:21:14.901Z" }, - { url = "https://files.pythonhosted.org/packages/e0/31/5158704cc474ab65c1647932e88be78dc0873f47130e253be38bcaf13d01/cffi-2.1.1-cp315-cp315-win_amd64.whl", hash = "sha256:e6e8cff14d6fb0be70a09c0bdc58096f501952d04624ebf867e0e56da2df8960", size = 187897, upload-time = "2026-08-03T21:21:16.108Z" }, - { url = "https://files.pythonhosted.org/packages/cc/4b/b3a2da8570c704ffc0f9762cdc3ec0f02c8573798e0b5cf7f11c82bbb70f/cffi-2.1.1-cp315-cp315-win_arm64.whl", hash = "sha256:27350daa11d4f10c540e6e89dada4c54feb7256ad03e9a4dc075ebad7ba360d1", size = 182935, upload-time = "2026-08-03T21:21:17.271Z" }, - { url = "https://files.pythonhosted.org/packages/d0/ef/5443574510a1207e6f6bc38ba6e1f1de36cb48fef07b2728bb896a21f430/cffi-2.1.1-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:c26608d2222fb1e94487e4a387d85f13eb55d5ed725cb25a0c589ac4ee60e7bc", size = 188464, upload-time = "2026-08-03T21:21:01.163Z" }, - { url = "https://files.pythonhosted.org/packages/7e/ae/a56fa8c4686ad50e148fcbc8d3ae0d03915ff5c30d795058988c24118cef/cffi-2.1.1-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:4be96343e422f2dfcd12ab5c9f5aebe03f82f737c6bffeca6830b3875cb44aab", size = 188262, upload-time = "2026-08-03T21:21:02.382Z" }, - { url = "https://files.pythonhosted.org/packages/53/b2/6187f46f2912276a3ae284076109cc5c8680482f11f766ccf26db4a86427/cffi-2.1.1-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:937c0052c05a31ca1daf18de3158eed4dbfcb9cc107adbea227728d647be701e", size = 223779, upload-time = "2026-08-03T21:21:03.553Z" }, - { url = "https://files.pythonhosted.org/packages/8a/f6/c3ad28bd19f77047a03084424fbd4cbe997303267c14423737324be0385d/cffi-2.1.1-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:df423d40ee8654634421812bc3b196da3f9bd7d32929da813f8394c4348a5358", size = 211520, upload-time = "2026-08-03T21:21:04.863Z" }, - { url = "https://files.pythonhosted.org/packages/a0/cd/ccac9013a5bd9fd764de118674ab9c805b5ca10c19270d90ee273f8b2240/cffi-2.1.1-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a730a083190634c65cca36ba5f489531576ebd79bcd5c8e172130f6453127231", size = 210673, upload-time = "2026-08-03T21:21:06.223Z" }, - { url = "https://files.pythonhosted.org/packages/52/86/2976131c639aead931c5bee5aba67e4b09fbeb8018b6f282f70803f923a7/cffi-2.1.1-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:363e05fa78e15116c3c32c210ee36884fd6b9afa6d440e47112c3bd511d64cb6", size = 223835, upload-time = "2026-08-03T21:21:07.539Z" }, - { url = "https://files.pythonhosted.org/packages/ac/0c/33a7aeab2f9c76918c52e084beb39c570db3588133412929e8ec06fab90b/cffi-2.1.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:770de9db11e84213beec501cfcaa013b019820ca881e03344dea5844f7876d94", size = 226705, upload-time = "2026-08-03T21:21:08.774Z" }, - { url = "https://files.pythonhosted.org/packages/e3/26/2cde30fdde421130bfc18f70395731a6e6b2053c6a1978a5258ff04e72fa/cffi-2.1.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:7da0c5eff80f0197f3b3d1232ec5a682a9325f4ae9016a78f5f5ca35f9ced1f5", size = 225539, upload-time = "2026-08-03T21:21:09.911Z" }, - { url = "https://files.pythonhosted.org/packages/6d/cd/a361394c94b2129d604bb846f624a8e88255a3ee33129c434a00d715e64f/cffi-2.1.1-cp315-cp315t-win32.whl", hash = "sha256:06c72bb76605a4b0cd0aad6930b69d4baf7dd5d806cfc409b824191099700e66", size = 182707, upload-time = "2026-08-03T21:21:11.226Z" }, - { url = "https://files.pythonhosted.org/packages/9b/b5/ba2b299993c26577d529b6ae29841f9e15b9fcf004d65f423f4fcf94ade9/cffi-2.1.1-cp315-cp315t-win_amd64.whl", hash = "sha256:d9c275eaacd24aa73f94ffd6de08fc3f932424d8b6c376f4bed7cde376fe7bc3", size = 193772, upload-time = "2026-08-03T21:21:12.39Z" }, - { url = "https://files.pythonhosted.org/packages/aa/29/35e016098c814cd93de9cd320c66b5bfba14dc6ecedd3cb518fa7c408c69/cffi-2.1.1-cp315-cp315t-win_arm64.whl", hash = "sha256:d18e5ac0f2f03f4f518d3e23db0f0cad7faa1da8620e9c09461d443bbf6e6692", size = 186360, upload-time = "2026-08-03T21:21:13.636Z" }, -] - [[package]] name = "cfgraph" version = "0.2.1" @@ -396,15 +302,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, ] -[[package]] -name = "comm" -version = "0.2.3" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/4c/13/7d740c5849255756bc17888787313b61fd38a0a8304fc4f073dfc46122aa/comm-0.2.3.tar.gz", hash = "sha256:2dc8048c10962d55d7ad693be1e7045d891b7ce8d999c97963a5e3e99c055971", size = 6319, upload-time = "2025-07-25T14:02:04.452Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/60/97/891a0971e1e4a8c5d2b20bbe0e524dc04548d2307fee33cdeba148fd4fc7/comm-0.2.3-py3-none-any.whl", hash = "sha256:c615d91d75f7f04f095b30d1c1711babd43bdc6419c1be9886a85f2f4e489417", size = 7294, upload-time = "2025-07-25T14:02:02.896Z" }, -] - [[package]] name = "coverage" version = "7.14.0" @@ -503,27 +400,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/9e/e9/bfa4d4a47c03180422343be8c69df75f5942276e815a367ae377f32db2d5/curies-0.14.6-py3-none-any.whl", hash = "sha256:a3653afa27576029c491e6ef97d2bc7d9c388afbbf44973e993b8d24fbb0c99c", size = 82629, upload-time = "2026-08-05T08:11:45.123Z" }, ] -[[package]] -name = "debugpy" -version = "1.8.21" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/f2/aa/12037145b7a56eaa5b29b41872f7a21b538e807e13f32c4d3c46e59be084/debugpy-1.8.21.tar.gz", hash = "sha256:a3c53278e84c94e11bd87c53970ec391d1a67396c8b22609fcac576520e611a6", size = 1697577, upload-time = "2026-06-01T19:30:35.156Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a2/df/bf625547431a9cadc9f4cbfeda38866e2b17f6aed147b625377e87834449/debugpy-1.8.21-cp312-cp312-macosx_15_0_universal2.whl", hash = "sha256:9f96713896f39c3dff0ee841f47320c3f2983d33c341e009361bb0ebc79adc4e", size = 2483609, upload-time = "2026-06-01T19:30:50.794Z" }, - { url = "https://files.pythonhosted.org/packages/bf/09/59324b903599031ff9faaec1758292409f6561a0ec2492fe4b703327705a/debugpy-1.8.21-cp312-cp312-manylinux_2_34_x86_64.whl", hash = "sha256:c193d474f0a211191f2b4449d2d06157c689013035bd952f3b617e0ef422b176", size = 3968900, upload-time = "2026-06-01T19:30:52.341Z" }, - { url = "https://files.pythonhosted.org/packages/14/cd/27f65b805d7fe005c44e1a36b9183ecdfbcdbf9d3e721a5115d461ecc7ee/debugpy-1.8.21-cp312-cp312-win32.whl", hash = "sha256:4743373c1cac7f9e74a1b9915bf1dbe0e900eca657ffb170ae07ac8363205ae9", size = 5336340, upload-time = "2026-06-01T19:30:54.047Z" }, - { url = "https://files.pythonhosted.org/packages/77/1d/c84e30c0c674184948b66f076ab271c01d940618a2824c23cd035a27bc20/debugpy-1.8.21-cp312-cp312-win_amd64.whl", hash = "sha256:bd7ba9dd3daa7c2f942c6ca8d4695a16bf9ac16b63615261c7982bc74f7ed20c", size = 5374751, upload-time = "2026-06-01T19:30:55.891Z" }, - { url = "https://files.pythonhosted.org/packages/77/6b/d817e1f8cc77aa055d37fba092e0febfdff40fe652d8d53d4cd7a86ad98d/debugpy-1.8.21-cp313-cp313-macosx_15_0_universal2.whl", hash = "sha256:13678151fc401e2d68c9880b91e28714f797d40422994572b24560ef80910a88", size = 2477398, upload-time = "2026-06-01T19:30:57.644Z" }, - { url = "https://files.pythonhosted.org/packages/48/57/412421516afc3055fa577516f00beec3d663f9b0ab330639547ae6c57720/debugpy-1.8.21-cp313-cp313-manylinux_2_34_x86_64.whl", hash = "sha256:ecbd158386c31ffe71d46f72d44d56e66331ab9b16cad649156d514368f23ab2", size = 3962096, upload-time = "2026-06-01T19:30:59.235Z" }, - { url = "https://files.pythonhosted.org/packages/c1/62/2c616337cf6ba7b07ebbc97f02c6c945a8e2f76b365e33ee809c32ee36d1/debugpy-1.8.21-cp313-cp313-win32.whl", hash = "sha256:2c2ae706dec41d99a9ca1f7ebc987a83e65578363be6f6b3ac9067504917fae1", size = 5336288, upload-time = "2026-06-01T19:31:00.79Z" }, - { url = "https://files.pythonhosted.org/packages/f8/99/9175103392f84c4b1bf7622888cdc68da07f0ff7d9e581266428f6776033/debugpy-1.8.21-cp313-cp313-win_amd64.whl", hash = "sha256:aa648733047443eb1d07682c4ef287d36a54507b643ffdf38b09a3ef002c72a0", size = 5376567, upload-time = "2026-06-01T19:31:02.56Z" }, - { url = "https://files.pythonhosted.org/packages/ce/3d/f4bbb323a548bfab2af3d6b4ffd9bf22636e55956a1285d317a1de643aad/debugpy-1.8.21-cp314-cp314-macosx_15_0_universal2.whl", hash = "sha256:9bb2a685287a2ac9b181cde89edcec64845cb51de7faaa75badb9a698bc24782", size = 2477209, upload-time = "2026-06-01T19:31:04.157Z" }, - { url = "https://files.pythonhosted.org/packages/8c/2d/6e7ec524984a1702777868de49a4c53202bddac2a432a76a093469587750/debugpy-1.8.21-cp314-cp314-manylinux_2_34_x86_64.whl", hash = "sha256:3d6922439bf33fd38a3e2c447869ebc7b97da5cd3d329ff1ef9bc06c4903437e", size = 3927115, upload-time = "2026-06-01T19:31:05.863Z" }, - { url = "https://files.pythonhosted.org/packages/97/47/d1aa6d64005a98a9144647d99306b419396f9ad7bf1d73c119e17a81fb4d/debugpy-1.8.21-cp314-cp314-win32.whl", hash = "sha256:15d4963bd5ffa48f0da0947fd06757fa7621945048a14ad7705431566d3c0e7c", size = 5336724, upload-time = "2026-06-01T19:31:07.711Z" }, - { url = "https://files.pythonhosted.org/packages/5f/67/b905b90d163af11878c1af8abafa4a25206335e112e284e413454543a6da/debugpy-1.8.21-cp314-cp314-win_amd64.whl", hash = "sha256:fe0744a12353406de0ae8ccff0d0a4a666f00801a3db8fd04e7a5f761cd520e8", size = 5373803, upload-time = "2026-06-01T19:31:09.469Z" }, - { url = "https://files.pythonhosted.org/packages/95/51/67e7cf11a53e40694f720457d5b3a1cdaaa3d5a9a633e482f225456b93ff/debugpy-1.8.21-py2.py3-none-any.whl", hash = "sha256:b1e37d333663c8851516a47364ef473da127f9caebe4417e6df6f5825a7e9a92", size = 5352888, upload-time = "2026-06-01T19:31:25.186Z" }, -] - [[package]] name = "decorator" version = "5.3.1" @@ -620,9 +496,7 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c4/37/4549f149c9797c21b32c2683c33522af22522099de128b2406672526d005/greenlet-3.5.1-cp312-cp312-macosx_11_0_universal2.whl", hash = "sha256:fa4f98af3a528f0c3fd592a26df7f376f93329c8f4d987f6bb979057af8bf5e2", size = 286220, upload-time = "2026-05-20T13:07:28.463Z" }, { url = "https://files.pythonhosted.org/packages/38/ff/a4f436709716965eaab9f36ea7b906c8a927fbe32fb1372a2071d964f6b1/greenlet-3.5.1-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ffea73584b216150eab159b6d12348fb253e68757974de1e2c40d8a318ac89ed", size = 601585, upload-time = "2026-05-20T14:00:06.141Z" }, { url = "https://files.pythonhosted.org/packages/65/ad/54bc3fcee3ad368a61b19b67d88117f7a8c29727bf71fffdeda81fbd946e/greenlet-3.5.1-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1072b4f9edcc1e192d9283a66a3e68d6b84c561de33a83d7858beb9ba1effe10", size = 614215, upload-time = "2026-05-20T14:05:42.675Z" }, - { url = "https://files.pythonhosted.org/packages/7c/6c/de5b1b388cd2d9fbdfeab324863daba37d54e6e233ddbefd70b385a8c591/greenlet-3.5.1-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:89101bfd5011e069be974903cb3a4e4523845e4ece2d62dcd8d358933c0ef249", size = 620094, upload-time = "2026-05-20T14:09:09.18Z" }, { url = "https://files.pythonhosted.org/packages/40/69/b91cda0647df839483201545913514c2827ebea5e5ccdf931842763bc127/greenlet-3.5.1-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:add5217d68b31130f0beca584d7fef4878327d2e31642b66618a14eef312b63b", size = 611358, upload-time = "2026-05-20T13:14:26.37Z" }, - { url = "https://files.pythonhosted.org/packages/4a/43/1204baffab8a6476464795a7ccf394a3248d4f22c9f87173a15b36b6d971/greenlet-3.5.1-cp312-cp312-manylinux_2_39_riscv64.whl", hash = "sha256:e6cd99ea59dd5d89f0c956606571d79bfe6f68c9eb7f4a4083a41a7f1587edee", size = 422782, upload-time = "2026-05-20T14:01:39.597Z" }, { url = "https://files.pythonhosted.org/packages/59/90/3cf77e080350cd02fa307bb2abf05df48f4482c240275bbd2c203ba8bb1c/greenlet-3.5.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a5ea42a752d47a145eae922b605cd1634665ac3d5ec1e72402d5048e8d60d207", size = 1570475, upload-time = "2026-05-20T14:02:25.29Z" }, { url = "https://files.pythonhosted.org/packages/65/2c/18cece62045e74598c3c393f70dce4a63f56222015ba29a5d4eeb04f764c/greenlet-3.5.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:c5551170cf4f5ff5623e9af81323751979fee2c731e2287b61f73cd27257b823", size = 1635625, upload-time = "2026-05-20T13:14:34.027Z" }, { url = "https://files.pythonhosted.org/packages/30/f5/310d104ddf41eb5a70f4c268d22508dfb0c3c8e86fec152be34d0d2ed819/greenlet-3.5.1-cp312-cp312-win_amd64.whl", hash = "sha256:3c8bb982ad117d29478ef8f5533e97df21f1e2befd17a299257b0c96d1371c0b", size = 238791, upload-time = "2026-05-20T13:10:39.018Z" }, @@ -630,9 +504,7 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/27/69/7f7e5372d998b81001899b1c0823c957aa413ba0f2662e65821611cc31e4/greenlet-3.5.1-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:51518ff74664078fc51bffcc6fc529b0df5ae58da192691cee765d45ce944a2b", size = 285060, upload-time = "2026-05-20T13:08:51.899Z" }, { url = "https://files.pythonhosted.org/packages/b1/bf/387f9b6b865fd2ae0d0be09e0004827295a01b71be76ed350dd1e28a91a4/greenlet-3.5.1-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ffdb3c0bb002c99cd8f298957e046c3dbf6006b5b7cdf11a4e19194624a0a0a", size = 604370, upload-time = "2026-05-20T14:00:07.492Z" }, { url = "https://files.pythonhosted.org/packages/32/f5/169ce3d4e4c67291bd18f8cbe0299c9f3e45102c7f1fb3c14780c93e4532/greenlet-3.5.1-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7715a5a2c3378ba602c3a440558261e13a820bb53a82693aacd7b7f6d964e283", size = 616987, upload-time = "2026-05-20T14:05:44.237Z" }, - { url = "https://files.pythonhosted.org/packages/19/ba/c24110c55dffa55aa6e1d98b45310da33801aeba7686ff0190fe5d46fd32/greenlet-3.5.1-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d40a890035c0058cadbdc4af7569800fd28a0e527a0fdbb7b5f9418f176846ce", size = 622911, upload-time = "2026-05-20T14:09:10.598Z" }, { url = "https://files.pythonhosted.org/packages/ee/e5/7f2e41d5273be07e77560d61ea4e56485b4d6c316d2a84518c62d1364061/greenlet-3.5.1-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:dc71ff466927a201b08305acac451ebe1aedfcea002f62f1f2f2ac2ac1e6a135", size = 613911, upload-time = "2026-05-20T13:14:27.539Z" }, - { url = "https://files.pythonhosted.org/packages/ec/7b/d20db2e8a5ad6c038702f3179b136f93f0a3d1a21a0c0777f3e470cdf4b2/greenlet-3.5.1-cp313-cp313-manylinux_2_39_riscv64.whl", hash = "sha256:67821bb03e4e98664490edb787ff6af501194c29bbee0f5c1dfdcf1dc3d9d436", size = 425228, upload-time = "2026-05-20T14:01:40.837Z" }, { url = "https://files.pythonhosted.org/packages/c5/a4/fbdc67579b73615a1f91615e814303cc71e06128f7baaba87be79b8fb90c/greenlet-3.5.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:cd443683db272ebaaca03af98c0b063ab30db70ea8a31a1559f35e3f7b744ccd", size = 1570689, upload-time = "2026-05-20T14:02:27.225Z" }, { url = "https://files.pythonhosted.org/packages/e6/b4/77abbe35078be39718a46cd49caf16bceb35662f97a34101dca28aa98e47/greenlet-3.5.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:089fff7a6ce8d9316d1f65ebc00273a56be258c1725b32b94de90a3a979557e1", size = 1635602, upload-time = "2026-05-20T13:14:36.344Z" }, { url = "https://files.pythonhosted.org/packages/37/f7/129f27ca700845b8ee8ca88ce7f43435a1239c2eddb7677fc938822762cf/greenlet-3.5.1-cp313-cp313-win_amd64.whl", hash = "sha256:110a1ca7b49b014b097f6078272c3f4ed31af45b254de5228b79adba879f6af9", size = 238683, upload-time = "2026-05-20T13:11:50.57Z" }, @@ -640,9 +512,7 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/8a/cb/c62454606daf5640369c94d8a9dd540599b1bfc090e2d2180cb77f4038d2/greenlet-3.5.1-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:d8ab31c9de8651a2facdd5c5bb0011f2380dd1a7af78ce2adf4b56095294fc07", size = 285579, upload-time = "2026-05-20T13:08:56.396Z" }, { url = "https://files.pythonhosted.org/packages/ec/71/c4270398c2eba968a6071af1dfbdcaeee6ec1c24bc8b435b8cc452700da6/greenlet-3.5.1-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e300185139abc337ade480c327183adf42a875ac7181bfe66d7d4efea31fbea", size = 651106, upload-time = "2026-05-20T14:00:09.448Z" }, { url = "https://files.pythonhosted.org/packages/1a/ab/71e34b78a44ec271fb5f550c17bc46d301ddc5953890d935f270b0dcdb5a/greenlet-3.5.1-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7ffdb990dcaa0234cf9845aead5df2e3c3a8b6507d409274dd87e0d5ab05ffc2", size = 663478, upload-time = "2026-05-20T14:05:45.88Z" }, - { url = "https://files.pythonhosted.org/packages/c6/2d/2d80842910da44f78c286532d084b8a5c3717c844ae80ceb3858738ae89a/greenlet-3.5.1-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6c09df69dc1712d131332054a858a3e5cca400967fa3a672e2324fbb0971448c", size = 667767, upload-time = "2026-05-20T14:09:12.15Z" }, { url = "https://files.pythonhosted.org/packages/77/96/4efd6fa5c62c85426a0c19077a586258ebc3a2a146ff2493e4312a697a22/greenlet-3.5.1-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2f82b3597e9d83b63408affed0b48fd0f54935edac4302237b9a837be0dae33c", size = 660800, upload-time = "2026-05-20T13:14:29.129Z" }, - { url = "https://files.pythonhosted.org/packages/e9/d3/dad2eecedfbb1ed7050a20dcfae40c1442b74bc7423608be2c7e03ee7133/greenlet-3.5.1-cp314-cp314-manylinux_2_39_riscv64.whl", hash = "sha256:a4764e0bfc6a4d114c865b32520805c16a990ef5f286a514413b05d5ecd6a23d", size = 470786, upload-time = "2026-05-20T14:01:42.064Z" }, { url = "https://files.pythonhosted.org/packages/7a/e0/6c71401a25cac7000261304e866a2f2cc04dc74810d40e2f118aa4799495/greenlet-3.5.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c0141e37414c10164e702b8fb1473304221ad98f71600850c6ef7ff4880feba0", size = 1617518, upload-time = "2026-05-20T14:02:28.662Z" }, { url = "https://files.pythonhosted.org/packages/41/26/c5c06643e8c0af9e7bf18e16cb51d0ab7625155f0392e1c9015d66d556cd/greenlet-3.5.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:50ae25a67bea74ea41fb14b960bc532df73eb713417b2d61892dced82fe8d3bc", size = 1681593, upload-time = "2026-05-20T13:14:39.417Z" }, { url = "https://files.pythonhosted.org/packages/8a/bd/e11a108317485075e68af9d23039619b86b28130c3b50d227d42edece64b/greenlet-3.5.1-cp314-cp314-win_amd64.whl", hash = "sha256:8a17c42330e261299766b75ac1ea32caa437a9453c8f65d16a13140db378ecd3", size = 239800, upload-time = "2026-05-20T13:09:30.128Z" }, @@ -650,18 +520,14 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/90/12/41bf27fde4d3605d3773ae57751eda182b8be2f5398011c041173b1d9534/greenlet-3.5.1-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:ea8da1e900d758d078810d4255d8c6aa572181896a31ec79d779eb79c3adc9ad", size = 293637, upload-time = "2026-05-20T13:12:35.529Z" }, { url = "https://files.pythonhosted.org/packages/44/44/ba14b23e9757707050c2f397d305bbcae62e5d7cad122f8b6baec5ae4a1f/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a19570c52a21420dcbc94e661994bc325c0b5b11304540fed514586da5dc8f2e", size = 650840, upload-time = "2026-05-20T14:00:11.079Z" }, { url = "https://files.pythonhosted.org/packages/a8/37/5ddc2b686a6844f91abecef43411842426da2e1573f60b49ecf2547f4ae1/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3d955c89b75eeca4723d7cc14135f393cd47c32e2a6cb4a8e4c6e760a26b0986", size = 656416, upload-time = "2026-05-20T14:05:47.118Z" }, - { url = "https://files.pythonhosted.org/packages/8c/46/5987dcd1a2570ba84f3b187536b2ca3ae97613387e57f5cfa99df068fe5e/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ea37d5a157eb9493820d3792ac4ece28619a394391d2b9f2f78057d396ff0f0f", size = 656607, upload-time = "2026-05-20T14:09:13.949Z" }, { url = "https://files.pythonhosted.org/packages/e1/f0/d17510297c35a2992712f0bf84de3779749999f7d3d63aa1f09db7c62dbe/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:de2daaaebd1a5aa88c49045b6baf9310b3263796bd88db713edf37cf53e7bb4e", size = 654397, upload-time = "2026-05-20T13:14:30.696Z" }, - { url = "https://files.pythonhosted.org/packages/2c/c1/6da0a9ddcc29d7e51ef14883fa3dc1e53b3f4ffba00582106c7bf55da1d8/greenlet-3.5.1-cp314-cp314t-manylinux_2_39_riscv64.whl", hash = "sha256:8d8a23250ea3ec7b36de8fa4b541e9e2db3ee82915cc060ab0631609ad8b28de", size = 488287, upload-time = "2026-05-20T14:01:43.143Z" }, { url = "https://files.pythonhosted.org/packages/37/eb/147387705bb89092645b012586e7273cb5ed3c90ef7eaf3a69173eaf0209/greenlet-3.5.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:3bfbd69cc349e43bf3a8ae1c85548ff0718efc887615c2db16c3833d7b0b072d", size = 1614469, upload-time = "2026-05-20T14:02:30.192Z" }, { url = "https://files.pythonhosted.org/packages/a6/4e/37ee0da7732b7aa9896f17e15579a9df34b9fcb9dd494f0adfa749af6623/greenlet-3.5.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:4378720dd888136c27215a0214d32a4d37c3852765d45bc37aad0623423cfd78", size = 1675115, upload-time = "2026-05-20T13:14:40.972Z" }, { url = "https://files.pythonhosted.org/packages/57/f3/97dfcf4a6eb5077f8a672234216fb5923eb89f2cab7081cb10b2cf75b605/greenlet-3.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:45718441607f9325d948db98cbc691276059316d0358c188c246da4e1d4d23d2", size = 245246, upload-time = "2026-05-20T13:12:22.646Z" }, { url = "https://files.pythonhosted.org/packages/5d/73/d7f72e34b582f694f4a9b248162db7b09cc458a259ba8f0c0bfa1a34ea7d/greenlet-3.5.1-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:2baee5ca02031757ffe8cc3d69f0cc0aec7065ce362622da74f32d3bcab1c541", size = 285575, upload-time = "2026-05-20T13:12:07.043Z" }, { url = "https://files.pythonhosted.org/packages/df/59/fa9c6e87dc8ad27a95dabe2f29f372b733d05a8a67470f6c901ed9975655/greenlet-3.5.1-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9b1ec3274918a81d3ea778b9e75b56b72b33f300edb6cf7f3a7fe1dae56683de", size = 656428, upload-time = "2026-05-20T14:00:12.556Z" }, { url = "https://files.pythonhosted.org/packages/f6/f9/e753408871eaa61dfe35e619cfc67512b036fde99893685d50eea9e07146/greenlet-3.5.1-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:111e2390ffffc47d5840b01711dd7fac07d4c09283d0283e7f3264b14e284c64", size = 667064, upload-time = "2026-05-20T14:05:48.662Z" }, - { url = "https://files.pythonhosted.org/packages/dc/74/807a047255bf1e09303627c46dc043dca596b6958a354d904f32ab382005/greenlet-3.5.1-cp315-cp315-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:10a9a1c0bfbc93d41156ffcb90c75fbc05544054faf15dcc1fdf9765f8b607f0", size = 672962, upload-time = "2026-05-20T14:09:15.532Z" }, { url = "https://files.pythonhosted.org/packages/96/27/5565b5b40389f1c7753003a07e21892fda8660926787036d5bc0308b8113/greenlet-3.5.1-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e630136e905fe5ff43e86945ae41220b6d1470956a39220e708110ac48d01ea5", size = 665697, upload-time = "2026-05-20T13:14:32.943Z" }, - { url = "https://files.pythonhosted.org/packages/76/32/19d4e13225193c29b13e308015223f7d75fd3d8623d49dd19040d2ce8ec1/greenlet-3.5.1-cp315-cp315-manylinux_2_39_riscv64.whl", hash = "sha256:ef08c1567c78074b22d1a200183d52d04a14df447bf70bcbb6a3507a48e776fc", size = 476047, upload-time = "2026-05-20T14:01:44.39Z" }, { url = "https://files.pythonhosted.org/packages/cf/82/e7de4178c0c2d1c9a5a3be3cc0b33e46a85b3ee4a77c071bf7ad8600e079/greenlet-3.5.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:975eac34b44a7077ca4d421348455b94f0f518246a7f14bc6d2fdcfe5b584368", size = 1621256, upload-time = "2026-05-20T14:02:31.91Z" }, { url = "https://files.pythonhosted.org/packages/00/10/f2dddcf7dacac17dfc68691809589adad06135eb28930429cf58a6467a2f/greenlet-3.5.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:9ab3c3a0b2ae6198e67c898dad5215a49f9ae0d0081b3c3ec59f333e39eeca26", size = 1685956, upload-time = "2026-05-20T13:14:42.55Z" }, { url = "https://files.pythonhosted.org/packages/22/17/4a232b32133230ada52f70e9d7f5b65b0caef8772f01849bd8d149e7e4ca/greenlet-3.5.1-cp315-cp315-win_amd64.whl", hash = "sha256:cbfc69be86e10dcfef5b1e6269d1d6926552aa89ee39e1de3353360c1b6989ab", size = 239802, upload-time = "2026-05-20T13:13:15.481Z" }, @@ -669,9 +535,7 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7a/57/816d9cff29119da3505b3d6a5e14a8af89006ac36f47f891ff293ee05af1/greenlet-3.5.1-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:a6fdf2433a5441ef9a95464f7c3e674775da1c8c1177fff311cee1acad4626ed", size = 293877, upload-time = "2026-05-20T13:10:19.078Z" }, { url = "https://files.pythonhosted.org/packages/23/a1/59b0a7c7d140ff1a75626680b9a9899b79a9176cab298b394968fb023295/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7546556f0d649f99f6a361098a55f761181bb2ea12ff150bb16d26092ad88244", size = 655333, upload-time = "2026-05-20T14:00:14.758Z" }, { url = "https://files.pythonhosted.org/packages/72/1b/5efe127597625042218939d01855109f352779050768b670b52edcc16a6c/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d5ee3ea898009fa898f85f9982255d35278c477bebe185beca249cab42d4526c", size = 659443, upload-time = "2026-05-20T14:05:50.159Z" }, - { url = "https://files.pythonhosted.org/packages/c9/9d/1dcdf7b95ab3cf8c7b6d7277c18a5e167312f2b362ddfcc5d5e6d8d84b43/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a57b0d05a0448eed231d59c0ceb287dde984551e54cbc51ac2d4865712838e9c", size = 659998, upload-time = "2026-05-20T14:09:16.912Z" }, { url = "https://files.pythonhosted.org/packages/6c/6d/c404246ea4d22d097a7426d0efb5b781bd7eb67715f09e79001bd552ab18/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a5c81f74d204d3edd136ebfd50dce53acbb776995d721a0fe801626cfc93b8cd", size = 658356, upload-time = "2026-05-20T13:14:35.091Z" }, - { url = "https://files.pythonhosted.org/packages/05/7e/c4959664fc231d587d66d8e81f2095e98056ba1954beafdcbe635e251052/greenlet-3.5.1-cp315-cp315t-manylinux_2_39_riscv64.whl", hash = "sha256:b0703c2cef53e01baec47f7a3868009913ad71ec678bbecb42a6f40895e4ce62", size = 494470, upload-time = "2026-05-20T14:01:45.611Z" }, { url = "https://files.pythonhosted.org/packages/51/02/f8ee37fb6d2219329f350af241c27fcf12df57e723d11f6fc6d3bacdadaa/greenlet-3.5.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:2c18ef16bf6d4dd410e4dd52996888ea1497be26892fe5bbc73580aba4287b8e", size = 1619216, upload-time = "2026-05-20T14:02:33.403Z" }, { url = "https://files.pythonhosted.org/packages/93/c5/3dc9475ace2c7a3680da12372cddd7f1ac874eb410a1ac48d3e9dab83782/greenlet-3.5.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:17d86354f0ae6b61bf9be5148d0dd34e06c3cb7c602c671f79f29ac3b150e659", size = 1678427, upload-time = "2026-05-20T13:14:43.71Z" }, { url = "https://files.pythonhosted.org/packages/df/4e/750c15c317a41ffb36f0bf40b933e3d744a7dede61889f74443ea69690cf/greenlet-3.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:e7516cf6ae6b8a582c2770a0caed47b8a48373ed732c33d69a72913ae6ac923e", size = 245225, upload-time = "2026-05-20T13:13:59.366Z" }, @@ -802,30 +666,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, ] -[[package]] -name = "ipykernel" -version = "7.3.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "appnope", marker = "sys_platform == 'darwin'" }, - { name = "comm" }, - { name = "debugpy" }, - { name = "ipython" }, - { name = "jupyter-client" }, - { name = "jupyter-core" }, - { name = "matplotlib-inline" }, - { name = "nest-asyncio2" }, - { name = "packaging" }, - { name = "psutil" }, - { name = "pyzmq" }, - { name = "tornado" }, - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/3d/c4/e4a38f579de4225a561305666f7541cdabb30075def2aa1ac17bd73c1fb5/ipykernel-7.3.0.tar.gz", hash = "sha256:9acaaaf97d16355166e4085afe9d225bfbdf2b7ef520f9df3be8f2b248275e09", size = 184899, upload-time = "2026-06-10T08:41:25.481Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/3d/02/77b271f5dc58bfbc0b577c877b2365d1ffea2afe66a80c13f2312820348c/ipykernel-7.3.0-py3-none-any.whl", hash = "sha256:897eb64da762549ef610698fca5e9675195ec6ac8ec7f19d81ce1ca20c876057", size = 120583, upload-time = "2026-06-10T08:41:23.648Z" }, -] - [[package]] name = "ipython" version = "9.13.0" @@ -1000,36 +840,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/41/45/1a4ed80516f02155c51f51e8cedb3c1902296743db0bbc66608a0db2814f/jsonschema_specifications-2025.9.1-py3-none-any.whl", hash = "sha256:98802fee3a11ee76ecaca44429fda8a41bff98b00a0f2838151b113f210cc6fe", size = 18437, upload-time = "2025-09-08T01:34:57.871Z" }, ] -[[package]] -name = "jupyter-client" -version = "8.9.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "jupyter-core" }, - { name = "python-dateutil" }, - { name = "pyzmq" }, - { name = "tornado" }, - { name = "traitlets" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/7d/dc/5512503b088997c2250b8bf18258fba9d9ce5ead641183700960d3c9d342/jupyter_client-8.9.1.tar.gz", hash = "sha256:a58f730dd9e728ba16ba1d62ebccf7ffe1ebbdbce4e95cfae941b7321ae1f4fa", size = 359256, upload-time = "2026-06-09T13:15:01.033Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/3f/6f/56d39bf385c5c27988aebaf0c18a2a17e960575740100973511018bd904e/jupyter_client-8.9.1-py3-none-any.whl", hash = "sha256:0b7a295bc46e8751e9adae84781f726c851c1d911bd793edc4a3bde942e3da81", size = 109828, upload-time = "2026-06-09T13:14:58.835Z" }, -] - -[[package]] -name = "jupyter-core" -version = "5.9.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "platformdirs" }, - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/02/49/9d1284d0dc65e2c757b74c6687b6d319b02f822ad039e5c512df9194d9dd/jupyter_core-5.9.1.tar.gz", hash = "sha256:4d09aaff303b9566c3ce657f580bd089ff5c91f5f89cf7d8846c3cdf465b5508", size = 89814, upload-time = "2025-10-16T19:19:18.444Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e7/e7/80988e32bf6f73919a113473a604f5a8f09094de312b9d52b79c2df7612b/jupyter_core-5.9.1-py3-none-any.whl", hash = "sha256:ebf87fdc6073d142e114c72c9e29a9d7ca03fad818c5d300ce2adc1fb0743407", size = 29032, upload-time = "2025-10-16T19:19:16.783Z" }, -] - [[package]] name = "linkml" version = "1.11.1" @@ -1334,15 +1144,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/32/28/79f0f8de97cce916d5ae88a7bee1ad724855e83e6019c0b4d5b3fabc80f3/mkdocstrings_python-2.0.3-py3-none-any.whl", hash = "sha256:0b83513478bdfd803ff05aa43e9b1fca9dd22bcd9471f09ca6257f009bc5ee12", size = 104779, upload-time = "2026-02-20T10:38:34.517Z" }, ] -[[package]] -name = "nest-asyncio2" -version = "1.7.2" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/b4/73/731debf26e27e0a0323d7bda270dc2f634b398e38f040a09da1f4351d0aa/nest_asyncio2-1.7.2.tar.gz", hash = "sha256:1921d70b92cc4612c374928d081552efb59b83d91b2b789d935c665fa01729a8", size = 14743, upload-time = "2026-02-13T00:34:04.386Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/c5/3c/3179b85b0e1c3659f0369940200cd6d0fa900e6cefcc7ea0bc6dd0e29ffb/nest_asyncio2-1.7.2-py3-none-any.whl", hash = "sha256:f5dfa702f3f81f6a03857e9a19e2ba578c0946a4ad417b4c50a24d7ba641fe01", size = 7843, upload-time = "2026-02-13T00:34:02.691Z" }, -] - [[package]] name = "numpy" version = "2.4.6" @@ -1449,6 +1250,7 @@ dependencies = [ { name = "rich" }, { name = "sqlalchemy" }, { name = "typer" }, + { name = "typing-extensions" }, ] [package.optional-dependencies] @@ -1483,8 +1285,8 @@ requires-dist = [ { name = "mkdocstrings-python", marker = "extra == 'dev'", specifier = ">=2.0.1" }, { name = "oa-configurator", specifier = ">=1.0.0,<2.0.0" }, { name = "oa-configurator", extras = ["dev", "postgres"], marker = "extra == 'dev'", specifier = ">=1.0.0,<2.0.0" }, - { name = "omop-semantics", marker = "extra == 'semantics'", specifier = ">=0.6.0" }, - { name = "orm-loader", specifier = ">=1.0.0,<2.0.0" }, + { name = "omop-semantics", marker = "extra == 'semantics'", specifier = ">=0.6.2" }, + { name = "orm-loader", specifier = ">=1.2.0,<2.0.0" }, { name = "pandas", specifier = ">=2.0" }, { name = "psycopg", extras = ["binary"], marker = "extra == 'postgres'", specifier = ">=3.2" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=9.0.3" }, @@ -1496,15 +1298,15 @@ requires-dist = [ { name = "sqlalchemy", specifier = ">=2.0.45" }, { name = "ty", marker = "extra == 'dev'", specifier = "==0.0.61" }, { name = "typer", specifier = ">=0.12" }, + { name = "typing-extensions", specifier = ">=4.5" }, ] provides-extras = ["dev", "postgres", "semantics"] [[package]] name = "omop-semantics" -version = "0.6.0" +version = "0.6.2" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "ipykernel" }, { name = "linkml" }, { name = "linkml-runtime" }, { name = "python-dotenv" }, @@ -1512,9 +1314,9 @@ dependencies = [ { name = "typing-extensions" }, { name = "urllib3" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/f7/48/728207d2067363675ec6be073bba14bb1523e23895e2cd63b8a7f04e62ca/omop_semantics-0.6.0.tar.gz", hash = "sha256:88043ffaf36feb1eb023816de14d8ea232529d6bdd3c2d67a40e283004369791", size = 203380, upload-time = "2026-08-10T08:15:44.584Z" } +sdist = { url = "https://files.pythonhosted.org/packages/c4/c8/89354f4c627a54490aa8d68175665f9f860c65be424e1efdffd16776adc5/omop_semantics-0.6.2.tar.gz", hash = "sha256:ba01bf589cbd7c68163390c07dfefd960780251635773d80a5a9f8243b2be8f5", size = 202242, upload-time = "2026-09-07T02:41:57.009Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b8/45/38c8620119cabba3681a906a66a9006cb6f217ea9c11e5c7acfdd36ef34f/omop_semantics-0.6.0-py3-none-any.whl", hash = "sha256:a0285d213e2b98923fc46cc9ca16aff539aea289ed6fd040e863aa12ffec2793", size = 66670, upload-time = "2026-08-10T08:15:43.179Z" }, + { url = "https://files.pythonhosted.org/packages/0a/a3/9b71a40c0a269dffb6188f38cc908e13e2d6290f92201587d45b2bafaf8b/omop_semantics-0.6.2-py3-none-any.whl", hash = "sha256:1a9ac1dd8ae2f31c8ee150b8fb9e644ee21d287b071e1d11b4258f6954bd91bb", size = 66803, upload-time = "2026-09-07T02:41:55.506Z" }, ] [[package]] @@ -1531,7 +1333,7 @@ wheels = [ [[package]] name = "orm-loader" -version = "1.0.0" +version = "1.2.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "chardet" }, @@ -1540,9 +1342,9 @@ dependencies = [ { name = "pyarrow" }, { name = "sqlalchemy" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/a9/f5/d71344e3f65dc29021e3e0b59d7ebfbd2070b11f8c9224cdb891278dc649/orm_loader-1.0.0.tar.gz", hash = "sha256:171969195afec304398ef8e3cceab52dba4b082347d0dc84feefff10b81bed52", size = 136810, upload-time = "2026-08-12T04:49:18.284Z" } +sdist = { url = "https://files.pythonhosted.org/packages/17/42/093713d63ac7af5b29d94fd33f1888f4d192b38c867b16349caa53c1c0af/orm_loader-1.2.0.tar.gz", hash = "sha256:a000527daab1aeaaeb460e135984330805d2243f5d603c8c649b7d0e2cf75ae4", size = 158420, upload-time = "2026-09-02T06:22:56.506Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/36/f8/952d95e9e6573787cd30f276604b1c2aad166e1922b9e486fff516a48919/orm_loader-1.0.0-py3-none-any.whl", hash = "sha256:3212894544fe1ae5e0f9e88ebf873addff9854726ec70a84f9140e39e591d68a", size = 56790, upload-time = "2026-08-12T04:49:16.697Z" }, + { url = "https://files.pythonhosted.org/packages/46/65/96fe27564dbb9b74fc1991a5ef3d057bf1cfa227f65bd742e83b2328cef0/orm_loader-1.2.0-py3-none-any.whl", hash = "sha256:068ab02e4f2767047aae02908bf50ef63f3951905da5b841a3ee6b931926ae8b", size = 69661, upload-time = "2026-09-02T06:22:55.183Z" }, ] [[package]] @@ -1647,7 +1449,7 @@ name = "pexpect" version = "4.9.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "ptyprocess", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, + { name = "ptyprocess" }, ] sdist = { url = "https://files.pythonhosted.org/packages/42/92/cc564bf6381ff43ce1f4d06852fc19a2f11d180f23dc32d9588bee2f149d/pexpect-4.9.0.tar.gz", hash = "sha256:ee7d41123f3c9911050ea2c2dac107568dc43b2d3b0c7557a33212c398ead30f", size = 166450, upload-time = "2023-11-25T09:07:26.339Z" } wheels = [ @@ -1859,15 +1661,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/51/be/6f79d55816d5c22557cf27533543d5d70dfe692adfbee4b99f2760674f38/pyarrow-24.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:c91d00057f23b8d353039520dc3a6c09d8608164c692e9f59a175a42b2ae0c19", size = 28131282, upload-time = "2026-04-21T10:51:16.815Z" }, ] -[[package]] -name = "pycparser" -version = "3.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz", hash = "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", size = 103492, upload-time = "2026-01-21T14:26:51.89Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" }, -] - [[package]] name = "pydantic" version = "2.13.4" @@ -2172,49 +1965,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/04/11/432f32f8097b03e3cd5fe57e88efb685d964e2e5178a48ed61e841f7fdce/pyyaml_env_tag-1.1-py3-none-any.whl", hash = "sha256:17109e1a528561e32f026364712fee1264bc2ea6715120891174ed1b980d2e04", size = 4722, upload-time = "2025-05-13T15:23:59.629Z" }, ] -[[package]] -name = "pyzmq" -version = "27.1.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "cffi", marker = "implementation_name == 'pypy'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/04/0b/3c9baedbdf613ecaa7aa07027780b8867f57b6293b6ee50de316c9f3222b/pyzmq-27.1.0.tar.gz", hash = "sha256:ac0765e3d44455adb6ddbf4417dcce460fc40a05978c08efdf2948072f6db540", size = 281750, upload-time = "2025-09-08T23:10:18.157Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/92/e7/038aab64a946d535901103da16b953c8c9cc9c961dadcbf3609ed6428d23/pyzmq-27.1.0-cp312-abi3-macosx_10_15_universal2.whl", hash = "sha256:452631b640340c928fa343801b0d07eb0c3789a5ffa843f6e1a9cee0ba4eb4fc", size = 1306279, upload-time = "2025-09-08T23:08:03.807Z" }, - { url = "https://files.pythonhosted.org/packages/e8/5e/c3c49fdd0f535ef45eefcc16934648e9e59dace4a37ee88fc53f6cd8e641/pyzmq-27.1.0-cp312-abi3-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:1c179799b118e554b66da67d88ed66cd37a169f1f23b5d9f0a231b4e8d44a113", size = 895645, upload-time = "2025-09-08T23:08:05.301Z" }, - { url = "https://files.pythonhosted.org/packages/f8/e5/b0b2504cb4e903a74dcf1ebae157f9e20ebb6ea76095f6cfffea28c42ecd/pyzmq-27.1.0-cp312-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3837439b7f99e60312f0c926a6ad437b067356dc2bc2ec96eb395fd0fe804233", size = 652574, upload-time = "2025-09-08T23:08:06.828Z" }, - { url = "https://files.pythonhosted.org/packages/f8/9b/c108cdb55560eaf253f0cbdb61b29971e9fb34d9c3499b0e96e4e60ed8a5/pyzmq-27.1.0-cp312-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:43ad9a73e3da1fab5b0e7e13402f0b2fb934ae1c876c51d0afff0e7c052eca31", size = 840995, upload-time = "2025-09-08T23:08:08.396Z" }, - { url = "https://files.pythonhosted.org/packages/c2/bb/b79798ca177b9eb0825b4c9998c6af8cd2a7f15a6a1a4272c1d1a21d382f/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:0de3028d69d4cdc475bfe47a6128eb38d8bc0e8f4d69646adfbcd840facbac28", size = 1642070, upload-time = "2025-09-08T23:08:09.989Z" }, - { url = "https://files.pythonhosted.org/packages/9c/80/2df2e7977c4ede24c79ae39dcef3899bfc5f34d1ca7a5b24f182c9b7a9ca/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_i686.whl", hash = "sha256:cf44a7763aea9298c0aa7dbf859f87ed7012de8bda0f3977b6fb1d96745df856", size = 2021121, upload-time = "2025-09-08T23:08:11.907Z" }, - { url = "https://files.pythonhosted.org/packages/46/bd/2d45ad24f5f5ae7e8d01525eb76786fa7557136555cac7d929880519e33a/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:f30f395a9e6fbca195400ce833c731e7b64c3919aa481af4d88c3759e0cb7496", size = 1878550, upload-time = "2025-09-08T23:08:13.513Z" }, - { url = "https://files.pythonhosted.org/packages/e6/2f/104c0a3c778d7c2ab8190e9db4f62f0b6957b53c9d87db77c284b69f33ea/pyzmq-27.1.0-cp312-abi3-win32.whl", hash = "sha256:250e5436a4ba13885494412b3da5d518cd0d3a278a1ae640e113c073a5f88edd", size = 559184, upload-time = "2025-09-08T23:08:15.163Z" }, - { url = "https://files.pythonhosted.org/packages/fc/7f/a21b20d577e4100c6a41795842028235998a643b1ad406a6d4163ea8f53e/pyzmq-27.1.0-cp312-abi3-win_amd64.whl", hash = "sha256:9ce490cf1d2ca2ad84733aa1d69ce6855372cb5ce9223802450c9b2a7cba0ccf", size = 619480, upload-time = "2025-09-08T23:08:17.192Z" }, - { url = "https://files.pythonhosted.org/packages/78/c2/c012beae5f76b72f007a9e91ee9401cb88c51d0f83c6257a03e785c81cc2/pyzmq-27.1.0-cp312-abi3-win_arm64.whl", hash = "sha256:75a2f36223f0d535a0c919e23615fc85a1e23b71f40c7eb43d7b1dedb4d8f15f", size = 552993, upload-time = "2025-09-08T23:08:18.926Z" }, - { url = "https://files.pythonhosted.org/packages/60/cb/84a13459c51da6cec1b7b1dc1a47e6db6da50b77ad7fd9c145842750a011/pyzmq-27.1.0-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:93ad4b0855a664229559e45c8d23797ceac03183c7b6f5b4428152a6b06684a5", size = 1122436, upload-time = "2025-09-08T23:08:20.801Z" }, - { url = "https://files.pythonhosted.org/packages/dc/b6/94414759a69a26c3dd674570a81813c46a078767d931a6c70ad29fc585cb/pyzmq-27.1.0-cp313-cp313-android_24_x86_64.whl", hash = "sha256:fbb4f2400bfda24f12f009cba62ad5734148569ff4949b1b6ec3b519444342e6", size = 1156301, upload-time = "2025-09-08T23:08:22.47Z" }, - { url = "https://files.pythonhosted.org/packages/a5/ad/15906493fd40c316377fd8a8f6b1f93104f97a752667763c9b9c1b71d42d/pyzmq-27.1.0-cp313-cp313t-macosx_10_15_universal2.whl", hash = "sha256:e343d067f7b151cfe4eb3bb796a7752c9d369eed007b91231e817071d2c2fec7", size = 1341197, upload-time = "2025-09-08T23:08:24.286Z" }, - { url = "https://files.pythonhosted.org/packages/14/1d/d343f3ce13db53a54cb8946594e567410b2125394dafcc0268d8dda027e0/pyzmq-27.1.0-cp313-cp313t-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:08363b2011dec81c354d694bdecaef4770e0ae96b9afea70b3f47b973655cc05", size = 897275, upload-time = "2025-09-08T23:08:26.063Z" }, - { url = "https://files.pythonhosted.org/packages/69/2d/d83dd6d7ca929a2fc67d2c3005415cdf322af7751d773524809f9e585129/pyzmq-27.1.0-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d54530c8c8b5b8ddb3318f481297441af102517602b569146185fa10b63f4fa9", size = 660469, upload-time = "2025-09-08T23:08:27.623Z" }, - { url = "https://files.pythonhosted.org/packages/3e/cd/9822a7af117f4bc0f1952dbe9ef8358eb50a24928efd5edf54210b850259/pyzmq-27.1.0-cp313-cp313t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6f3afa12c392f0a44a2414056d730eebc33ec0926aae92b5ad5cf26ebb6cc128", size = 847961, upload-time = "2025-09-08T23:08:29.672Z" }, - { url = "https://files.pythonhosted.org/packages/9a/12/f003e824a19ed73be15542f172fd0ec4ad0b60cf37436652c93b9df7c585/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c65047adafe573ff023b3187bb93faa583151627bc9c51fc4fb2c561ed689d39", size = 1650282, upload-time = "2025-09-08T23:08:31.349Z" }, - { url = "https://files.pythonhosted.org/packages/d5/4a/e82d788ed58e9a23995cee70dbc20c9aded3d13a92d30d57ec2291f1e8a3/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:90e6e9441c946a8b0a667356f7078d96411391a3b8f80980315455574177ec97", size = 2024468, upload-time = "2025-09-08T23:08:33.543Z" }, - { url = "https://files.pythonhosted.org/packages/d9/94/2da0a60841f757481e402b34bf4c8bf57fa54a5466b965de791b1e6f747d/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:add071b2d25f84e8189aaf0882d39a285b42fa3853016ebab234a5e78c7a43db", size = 1885394, upload-time = "2025-09-08T23:08:35.51Z" }, - { url = "https://files.pythonhosted.org/packages/4f/6f/55c10e2e49ad52d080dc24e37adb215e5b0d64990b57598abc2e3f01725b/pyzmq-27.1.0-cp313-cp313t-win32.whl", hash = "sha256:7ccc0700cfdf7bd487bea8d850ec38f204478681ea02a582a8da8171b7f90a1c", size = 574964, upload-time = "2025-09-08T23:08:37.178Z" }, - { url = "https://files.pythonhosted.org/packages/87/4d/2534970ba63dd7c522d8ca80fb92777f362c0f321900667c615e2067cb29/pyzmq-27.1.0-cp313-cp313t-win_amd64.whl", hash = "sha256:8085a9fba668216b9b4323be338ee5437a235fe275b9d1610e422ccc279733e2", size = 641029, upload-time = "2025-09-08T23:08:40.595Z" }, - { url = "https://files.pythonhosted.org/packages/f6/fa/f8aea7a28b0641f31d40dea42d7ef003fded31e184ef47db696bc74cd610/pyzmq-27.1.0-cp313-cp313t-win_arm64.whl", hash = "sha256:6bb54ca21bcfe361e445256c15eedf083f153811c37be87e0514934d6913061e", size = 561541, upload-time = "2025-09-08T23:08:42.668Z" }, - { url = "https://files.pythonhosted.org/packages/87/45/19efbb3000956e82d0331bafca5d9ac19ea2857722fa2caacefb6042f39d/pyzmq-27.1.0-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:ce980af330231615756acd5154f29813d553ea555485ae712c491cd483df6b7a", size = 1341197, upload-time = "2025-09-08T23:08:44.973Z" }, - { url = "https://files.pythonhosted.org/packages/48/43/d72ccdbf0d73d1343936296665826350cb1e825f92f2db9db3e61c2162a2/pyzmq-27.1.0-cp314-cp314t-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:1779be8c549e54a1c38f805e56d2a2e5c009d26de10921d7d51cfd1c8d4632ea", size = 897175, upload-time = "2025-09-08T23:08:46.601Z" }, - { url = "https://files.pythonhosted.org/packages/2f/2e/a483f73a10b65a9ef0161e817321d39a770b2acf8bcf3004a28d90d14a94/pyzmq-27.1.0-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7200bb0f03345515df50d99d3db206a0a6bee1955fbb8c453c76f5bf0e08fb96", size = 660427, upload-time = "2025-09-08T23:08:48.187Z" }, - { url = "https://files.pythonhosted.org/packages/f5/d2/5f36552c2d3e5685abe60dfa56f91169f7a2d99bbaf67c5271022ab40863/pyzmq-27.1.0-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01c0e07d558b06a60773744ea6251f769cd79a41a97d11b8bf4ab8f034b0424d", size = 847929, upload-time = "2025-09-08T23:08:49.76Z" }, - { url = "https://files.pythonhosted.org/packages/c4/2a/404b331f2b7bf3198e9945f75c4c521f0c6a3a23b51f7a4a401b94a13833/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:80d834abee71f65253c91540445d37c4c561e293ba6e741b992f20a105d69146", size = 1650193, upload-time = "2025-09-08T23:08:51.7Z" }, - { url = "https://files.pythonhosted.org/packages/1c/0b/f4107e33f62a5acf60e3ded67ed33d79b4ce18de432625ce2fc5093d6388/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:544b4e3b7198dde4a62b8ff6685e9802a9a1ebf47e77478a5eb88eca2a82f2fd", size = 2024388, upload-time = "2025-09-08T23:08:53.393Z" }, - { url = "https://files.pythonhosted.org/packages/0d/01/add31fe76512642fd6e40e3a3bd21f4b47e242c8ba33efb6809e37076d9b/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cedc4c68178e59a4046f97eca31b148ddcf51e88677de1ef4e78cf06c5376c9a", size = 1885316, upload-time = "2025-09-08T23:08:55.702Z" }, - { url = "https://files.pythonhosted.org/packages/c4/59/a5f38970f9bf07cee96128de79590bb354917914a9be11272cfc7ff26af0/pyzmq-27.1.0-cp314-cp314t-win32.whl", hash = "sha256:1f0b2a577fd770aa6f053211a55d1c47901f4d537389a034c690291485e5fe92", size = 587472, upload-time = "2025-09-08T23:08:58.18Z" }, - { url = "https://files.pythonhosted.org/packages/70/d8/78b1bad170f93fcf5e3536e70e8fadac55030002275c9a29e8f5719185de/pyzmq-27.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:19c9468ae0437f8074af379e986c5d3d7d7bfe033506af442e8c879732bedbe0", size = 661401, upload-time = "2025-09-08T23:08:59.802Z" }, - { url = "https://files.pythonhosted.org/packages/81/d6/4bfbb40c9a0b42fc53c7cf442f6385db70b40f74a783130c5d0a5aa62228/pyzmq-27.1.0-cp314-cp314t-win_arm64.whl", hash = "sha256:dc5dbf68a7857b59473f7df42650c621d7e8923fb03fa74a526890f4d33cc4d7", size = 575170, upload-time = "2025-09-08T23:09:01.418Z" }, -] - [[package]] name = "rdflib" version = "7.6.0" @@ -2697,23 +2447,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c7/18/c86eb8e0202e32dd3df50d43d7ff9854f8e0603945ff398974c1d91ac1ef/tomli_w-1.2.0-py3-none-any.whl", hash = "sha256:188306098d013b691fcadc011abd66727d3c414c571bb01b1a174ba8c983cf90", size = 6675, upload-time = "2025-01-15T12:07:22.074Z" }, ] -[[package]] -name = "tornado" -version = "6.5.7" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/64/24/95ec527ad67b76d59299e5465b3935d05e4294b7e0290a3924b7487df30b/tornado-6.5.7.tar.gz", hash = "sha256:66c513a76cda70d53907bc27cf1447557699c2e95aa48ba27a442ff61c3ddfc2", size = 519252, upload-time = "2026-06-08T17:34:51.232Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/02/dc/c7043cab6fed8ae159fc1923ce829ada35c4dbd797d408a43858ffaf9639/tornado-6.5.7-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:148b2eb15c2c765a50796172c1e499649b35f30d2e3c3d3e15913cfa56bfb163", size = 448543, upload-time = "2026-06-08T17:34:38.052Z" }, - { url = "https://files.pythonhosted.org/packages/92/4f/090b1431e5a43df696feceffc268c5383cc079ecb5f08ce58f917109aafe/tornado-6.5.7-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:9da38de27f1da3b78a966f0dae12b5a1ea9afe72ca805d84ff06508272ddf100", size = 446707, upload-time = "2026-06-08T17:34:39.594Z" }, - { url = "https://files.pythonhosted.org/packages/37/d8/ef374952fd5da67d4463122c2b8e5a96536ec10b4b339254c6dcde81d01c/tornado-6.5.7-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:8d759e71906ee783f8867b93bf26a265743da4c1e2f4a018464c1ba019862972", size = 449774, upload-time = "2026-06-08T17:34:41.204Z" }, - { url = "https://files.pythonhosted.org/packages/35/37/d434c73f4c6e014b745b9b37085f34f40c022f007efff3d7fe65991899f3/tornado-6.5.7-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8a46347a18f23fb92b396beebe0fb78f61dda0cc302445202c16203d8a18848b", size = 450745, upload-time = "2026-06-08T17:34:42.531Z" }, - { url = "https://files.pythonhosted.org/packages/b6/2b/56b9aff361d7f1ab728a805ec7d7ea835f8807afa9f5cc690ea0e630efb9/tornado-6.5.7-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:7778b30bef919231265e91c69963ce0f49a1e9c07ac900bbe75b19ce2575ba92", size = 450578, upload-time = "2026-06-08T17:34:43.787Z" }, - { url = "https://files.pythonhosted.org/packages/02/30/a7444fb23aa76860a14198fab96ac79f1866b0a6e19e26c4381b0938e50f/tornado-6.5.7-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:e726f0c75da7726eec023aa62751ff8878bd2737e34fbdd33b1ae5897d2200f5", size = 449985, upload-time = "2026-06-08T17:34:45.326Z" }, - { url = "https://files.pythonhosted.org/packages/5c/42/5f0e56c01e8d9d36f4e23f367b85ae6cae0c1ecddd5e6977d8388ad27488/tornado-6.5.7-cp39-abi3-win32.whl", hash = "sha256:f8de3bf12d3efdd0cbe7c8887868198f8a91415e3f29fcf258d9b8eb7b1d9ae4", size = 451047, upload-time = "2026-06-08T17:34:46.784Z" }, - { url = "https://files.pythonhosted.org/packages/c9/a4/b393076ffb21b469eec5b328a0534cf03a3b90bfc6b1f09507cdd075d938/tornado-6.5.7-cp39-abi3-win_amd64.whl", hash = "sha256:de942f843533a039ef9fa3d9c88c7cd8a7c94553fb5ad0154270989b3d99a2c4", size = 451485, upload-time = "2026-06-08T17:34:48.248Z" }, - { url = "https://files.pythonhosted.org/packages/71/2e/7b1c769803121b809112cf9a00681c472eae1d80e32d7ec0e0bd61d0d0e1/tornado-6.5.7-cp39-abi3-win_arm64.whl", hash = "sha256:ff934fce95643af5f11efdae618eaa73d469dc588641e5c8d19295a0c65c4796", size = 450506, upload-time = "2026-06-08T17:34:49.702Z" }, -] - [[package]] name = "tqdm" version = "4.70.0"