AustralianCancerDataNetwork · nicoloesch · Sep 1, 2026 · Sep 1, 2026 · Sep 3, 2026 · Sep 3, 2026
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -22,7 +22,7 @@ jobs:
     with:
       ty-src: omop_alchemy
   build-test-postgres:
-    uses: AustralianCancerDataNetwork/cava-devops/.github/workflows/build-test-postgres.yml@main
+    uses: AustralianCancerDataNetwork/cava-devops/.github/workflows/build-test-postgres-v2.yml@main
     with:
       ty-src: omop_alchemy
       postgres-db: omop_alchemy_ci
@@ -43,14 +43,11 @@ jobs:
           --password test \
           --database-name omop_alchemy_test_ci \
           --test-only true
-        uv run omop-config databases add cdm_db \
-          --kind cdm \
+        uv run omop-config databases add cdm cdm_db \
           --connection ci_pg \
-          --schema-name public
-        uv run omop-config databases add test_cdm_db \
-          --kind cdm \
+          --cdm-schema public
+        uv run omop-config databases add cdm test_cdm_db \
           --connection ci_pg_test \
-          --schema-name public
+          --cdm-schema public
         uv run omop-config configure omop_alchemy \
-          --cdm-db cdm_db \
-          --test-cdm-db test_cdm_db
+          --test-cdm-db-pg test_cdm_db
diff --git a/Dockerfile b/Dockerfile
diff --git a/docs/advanced/backends.md b/docs/advanced/backends.md
@@ -0,0 +1,41 @@
+# Backend Compatibility
+
+Every maintenance operation goes through a `Backend` (`PostgresBackend` or
+`SQLiteBackend`). PostgreSQL implements the full `Backend` interface.
+SQLite implements only what SQLite itself can do; everything else raises
+`FeatureNotSupportedError` at call time rather than failing silently or
+producing a partial result.
+
+| Feature | PostgreSQL | SQLite |
+| --- | --- | --- |
+| Index existence check | Yes | Yes |
+| Drop index if exists | Yes | Yes (unqualified — no schema concept) |
+| `ANALYZE` | Yes | Yes |
+| `VACUUM ANALYZE` | Yes | No |
+| FK trigger management / status | Yes | No |
+| FK constraint violation counting | Yes | No |
+| Table clustering (`CLUSTER`) | Yes | No |
+| Cluster index inspection | Yes | No |
+| Functional-index expression normalization | Yes | No (SQLite never reflects expression-based indexes) |
+| `TRUNCATE ... RESTART IDENTITY / CASCADE` | Yes | No |
+| Sequence lookup / reset | Yes | No (SQLite has no sequences) |
+| Full-text search | Yes — see [PostgreSQL Full-Text Search](fulltext.md) | No |
+| Database backup / restore | Yes | No |
+
+## What this means in practice
+
+Maintenance CLI commands that rely on a not-supported feature raise
+`FeatureNotSupportedError` on SQLite rather than doing nothing. In
+particular, against a SQLite database:
+
+- `indexes cluster` and the clustering step of `manage_indexes --enable`
+  are unavailable.
+- `truncate-tables` cannot use `RESTART IDENTITY`/`CASCADE`.
+- `fulltext install` is unavailable entirely.
+- `backup-database`/`restore-database` are unavailable entirely.
+- FK trigger toggling and FK violation counting are unavailable.
+
+SQLite remains fully supported for the core ORM/CDM layer (models, queries,
+`create_missing_tables`, schema-provenance guarding) — these limitations are
+specific to the maintenance operations listed above, which assume a
+PostgreSQL-grade catalog.
diff --git a/docs/advanced/index.md b/docs/advanced/index.md
@@ -17,3 +17,4 @@ the immutability and interpretability of the underlying CDM tables.
 
 - [Backend Compatibility](backends.md)
 - [PostgreSQL Full-Text Search](fulltext.md)
+- [Vocabulary Load Performance](vocabulary_load_performance.md)
diff --git a/docs/cli/index.md b/docs/cli/index.md
@@ -90,7 +90,7 @@ With the decorator, the function body is all that matters:
 @app.command("my-command")
 @omop_command("my-command")
 def my_command(conn, engine) -> None:
-    results = do_work(engine, db_schema=conn.db_schema)
+    results = do_work(engine, db_schema=conn.resolved.schema_name)
     console.print(render_results(results))
 ```
 
@@ -102,5 +102,5 @@ def my_command(conn, engine) -> None:
 
 | Attribute | Description |
 |---|---|
-| `conn.db_schema` | CDM schema name from the resolved database (e.g. `"omop"`) |
+| `conn.resolved` | The resolved `ResolvedCDMDatabase`; `conn.resolved.schema_name` is the CDM schema name (e.g. `"omop"`) |
 | `conn.athena_source` | Athena vocabulary CSV directory from `[tools.omop_alchemy]`'s `athena_source_path` field; `None` if not configured |
diff --git a/docs/getting-started/common-use-cases.md b/docs/getting-started/common-use-cases.md
@@ -0,0 +1,130 @@
+# Common Use Cases
+
+This pages details common use-cases and setups for users and how to wrap their established OMOP CDM with `omop-alchemy` and configure it with `oa-configurator`. 
+
+!!! note "Important References"
+    - [**`oa-configurator` config reference**](https://AustralianCancerDataNetwork.github.io/oa-configurator/config-reference/): Information about the config file created and stored by default at `~/.config/omop/config.toml`
+    - [**`omop-alchemy` and relationship to OMOP CDM**](configuration.md#cdm-table-roles): Important how schemas capture specific tables.
+    - [**`oa_configurator`'s architecture guide`**](https://AustralianCancerDataNetwork.github.io/oa-configurator/architecture/): Information about the core concepts, supported configuration templates, schema translation and provenance guard, and more.
+    - [**`omop-alchemy`'s maintenance module**](maintenance.md): full command reference 
+
+
+---
+
+## Vocabulary tables in a separate schema, same server
+
+!!! example "Scenario"
+    - Your `Concept` table (and the rest of the vocabulary) lives in a `myvocab` schema
+    - Vocabulary is separate from your clinical tables' schema. 
+    - [Reading how `omop-alchemy` bundles tables in schemas](configuration.md#cdm-table-roles) reveals, that the `vocab_schema` configuration key is responsible for the `Concept` table
+
+### Solution
+Set `vocab_schema` to `myvocab` during interactive configuration
+```bash
+omop-config configure omop_alchemy
+```
+
+The resulting entry in `config.toml` will look like this:
+
+```toml
+[connections.<your-configured-connection>]
+dialect = ...
+host = ...
+....
+
+[databases.<your-configured-database>]
+kind = "cdm"
+connection = "<your-configured-connection>"
+cdm_schema = "<regular schema name>"
+vocab_schema = "myvocab"  # <- overwritten schema map
+
+[tools.omop_alchemy]
+cdm_db = "<your-configured-database>"
+```
+
+Every vocabulary-tagged table (see [Documentation for more details](configuration.md#cdm-table-roles)) now resolves into `myvocab` automatically.
+This does not require any model changes. `results_schema` works the same way for results tables and is also defined in the [Documentation](configuration.md#cdm-table-roles)
+
+### Troubleshooting
+
+#### 1. You misconfigured the schema wrong for your select database
+
+No issues. Just re-run the configuration command again:
+```bash
+omop-config configure omop_alchemy
+```
+
+The CLI wizard will guide you through the entire setup again. You can changed/modify settings. Previously configured fields are now the default and can just be accepted by pressing 'Enter'.
+
+---
+
+## Vocabulary on an entirely separate server
+
+!!! example "Scenario"
+    - Your entire CDM vocabulary lives on a separate physical DB server (e.g. a shared vocabnulary instance resued across multiple CDM deployments)
+    - You checked the documentation for [supported dialects in `omop-alchemy`](https://AustralianCancerDataNetwork.github.io/oa-configurator/config-reference/#supported-dialects ) and confirmed that your separate DB server is supported
+
+
+### Solution
+
+Configure a separate `vocab_connection` during the setup of `omop-alchemy` when prompted:
+```bash
+omop-config configure omop_alchemy
+```
+
+```toml
+[connections.cdm]  # <- your CDM connection
+dialect       = "postgresql+psycopg"
+host          = "cdm-db.internal"
+database_name = "cdm"
+
+[connections.vocab]  # <- your vocab connection
+dialect       = "postgresql+psycopg"
+host          = "vocab-db.internal"
+database_name = "vocab"
+
+[databases.cdm_db]
+kind             = "cdm"
+connection       = "cdm"
+vocab_connection = "vocab"  # <- references your vocabulary DB
+```
+
+!!! warning "Queries spanning both databases"
+    Reads that only touch vocabulary tables route to the `vocab` connection automatically. However, a query joining a vocabulary table against a clinical table (that lives in the `cdm` connection/DB) can't be answered by one physical connection in the underlying backend `SQLAlchemy`. `omop-alchemy` resolves those by querying each side separately and then merging the results in Python. Large queries therefore require significant memory budgets depending on the query.
+
+---
+
+## Migrating an existing deployment to a schema split
+
+!!! example "Scenario"
+    - You are moving from one schema holding everything to a real vocabulary/results splits, **or**
+    - You are renaming a schema on a database `omop-alchemy` has already created tables in.
+    - **Assumptions:**
+        - your database for the CDM is named `my_db` in `config.toml`
+            - there is an entry called `[databases.my_db]`, and
+            - `[tools.omop_alchemy]` lists it as `cdm_db="my_db"`
+        - you want to move all your tables governed by the interal `vocab` schema to schema `myvocab`
+
+### Solution
+
+[`oa-configurator`'s schema provenance guard](https://AustralianCancerDataNetwork.github.io/oa-configurator/architecture/#schema-provenance-guard) records which physical schema each role last resolved to, and refuses to run `create-missing-tables` if the configured schema for a role has silently changed since the last run. This mechanism is in place to stop a misconfiguration from creating an orphaned second copy of your tables. To make a genuine change deliberately:
+
+1. Update `vocab_schema`/`results_schema`/`cdm_schema` in `config.toml` through reconfiguration  
+    ```bash
+    omop-config configure omop_alchemy
+    ```
+2. Move your actual data to the new schema yourself using access to the database.
+    - This is **NEVER** done automatically to preserve data integrity from our end.
+3. Record the new schema as the accepted baseline following the assumptions listed in "Scenario" above:
+   ```bash
+   omop-config acknowledge-schema-migration --database my_db --schema-tag vocab --new-schema myvocab --reason "moving vocab off the shared schema"
+   ```
+4. Once you've confirmed the new schema is correct, clean up the old one:
+   ```bash
+   omop-config drop-orphan-schema-tables --database cdm_db --schema old_vocab_schema --confirm
+   ```
+   Omit `--confirm` first to preview what would be dropped.
+
+   Both commands live in `oa-configurator`, not `omop-alchemy` as they're generic over any `[databases.*]` entry, not CDM-specific.
+
+
diff --git a/docs/getting-started/configuration.md b/docs/getting-started/configuration.md
@@ -29,16 +29,39 @@ database_name = "omop_cdm"
 test_only     = false
 
 [databases.cdm_db]
-kind        = "cdm"
-connection  = "cdm"
-schema_name = "omop"
+kind       = "cdm"
+connection = "cdm"
+cdm_schema = "omop"
 
 [tools.omop_alchemy]
 cdm_db = "cdm_db"
 ```
 
 You can also write or edit this file manually. It follows the `oa-configurator` pattern of [physical]->[logical] resource definition, where one connection may serve multiple databases, and each application may define its own database resource, or choose to cross reference an existing one that will be resolved upon connection in the consuming application.
 
+## CDM table roles
+
+OMOP_Alchemy tags every table with a logical role, matching the [OMOP CDM v5.4](https://ohdsi.github.io/CommonDataModel/cdm54.html)
+categories:
+
+- **Clinical/derived tables** (`Role.PRIMARY`):
+    - All other tables not captured by the configurations below
+    - Controlled by `cdm_schema` in the configuration.
+- **Vocabulary tables** (`Role.VOCAB`):
+    - `concept`, `concept_ancestor`, `concept_class`, `concept_relationship`, `concept_synonym`, `domain`, `drug_strength`, `relationship`, `source_to_concept_map`, `vocabulary`
+    - Controlled by `vocab_schema` in the configuration.
+- **Results/analytics tables** (`Role.RESULTS`):
+    - `cohort`, `cohort_definition`
+    - Controlled by `results_schema` in the configuration.
+
+![OMOP CDM v5.4](https://ohdsi.github.io/CommonDataModel/man/images/cdm55.png)
+
+Each role folds back to `cdm_schema` when its own field is unset, so a minimal config needs no extra fields. 
+Setting `vocab_schema`/`results_schema` routes just that role's tables elsewhere. See [oa_configurator's schema translate map guide](https://AustralianCancerDataNetwork.github.io/oa-configurator/architecture/#schema-translate-map) for how the routing itself works, and [Common Use Cases](common-use-cases.md) for worked examples of splitting these onto different schemas or servers.
+
+!!! info "Misconfiguration prevention"
+    Misconfiguring which schema a role points at doesn't corrupt data. [`oa-configurator`'s schema provenance guard](https://AustralianCancerDataNetwork.github.io/oa-configurator/architecture/#schema-provenance-guard) refuses the DDL.
+
 ## Vocabulary loading
 
 If you plan to load OMOP vocabulary from Athena CSV files, add the path to `[tools.omop_alchemy]`:
@@ -76,5 +99,6 @@ See the [oa-configurator integration guide](https://AustralianCancerDataNetwork.
 
 ## Further reading
 
+- [Common Use Cases](common-use-cases.md): worked examples for vocab/results schema splits, a separate vocabulary server, and migrating an existing deployment's schema layout
 - [oa_configurator quickstart](https://AustralianCancerDataNetwork.github.io/oa-configurator/quickstart/): full config reference, CLI walkthrough
 - [oa_configurator integration guide](https://AustralianCancerDataNetwork.github.io/oa-configurator/integration/): multi-package setups
diff --git a/docs/getting-started/index.md b/docs/getting-started/index.md
@@ -9,5 +9,7 @@ These pages cover installation, maintenance tooling, and a minimal quickstart fo
 ## Orientation
 
 - [Installation](installation.md)
+- [Configuration](configuration.md)
 - [Maintenance CLI](maintenance.md)
+- [Common Use Cases](common-use-cases.md)
 - [Quickstart](quickstart.md)
diff --git a/docs/getting-started/maintenance.md b/docs/getting-started/maintenance.md
@@ -16,7 +16,7 @@ Some commands depend on PostgreSQL-specific features and will return an error if
 
 | Command group | Requires PostgreSQL | Why |
 | --- | --- | --- |
-| `load-vocab-source` | No (PostgreSQL + SQLite) | Uses ORM CSV loader; `--bulk-mode` and `--db-schema` are PostgreSQL-only |
+| `load-vocab-source` | No (PostgreSQL + SQLite) | Uses ORM CSV loader; `--bulk-mode` is PostgreSQL-only |
 | `indexes` | No (cluster apply is PostgreSQL-only) | Index DDL is standard SQL; `CLUSTER` is PostgreSQL |
 | `create-missing-tables`, `reconcile-schema`, `data-summary`, `info`, `doctor` | No | Pure SQLAlchemy metadata operations |
 | `reset-sequences` | Yes | PostgreSQL sequences (`SETVAL`) |
@@ -271,6 +271,13 @@ omop-alchemy reset-sequences --vocab  # vocabulary tables only
 
 ---
 
+!!! note "Schema-drift protection"
+    `fulltext install`, `indexes enable`/`disable`/`cluster`, `truncate-tables`, and
+    vocabulary-table creation each guard their DDL against the configured schema having
+    silently drifted since it was last recorded, and raise `SchemaDriftError` if it has.
+    See [Schema drift](#schema-drift) below for the remediation path
+    (`omop-config acknowledge-schema-migration`).
+
 ## Command reference
 
 | Command | Purpose | Key options | Backend |
@@ -290,6 +297,7 @@ omop-alchemy reset-sequences --vocab  # vocabulary tables only
 | `analyze-tables` | Refresh planner statistics | `--scope`, `--table`, `--vacuum` | PostgreSQL, SQLite (`--vacuum` PostgreSQL-only) |
 | `indexes disable` | Drop ORM-defined secondary indexes | `--vocab`, `--dry-run` | All |
 | `indexes enable` | Recreate ORM-defined secondary indexes | `--vocab`, `--dry-run` | All (cluster on PostgreSQL) |
+| `indexes cluster` | Physically rewrite tables sorted by their cluster index | `--vocab`, `--dry-run` | PostgreSQL |
 | `fulltext install` | Add tsvector sidecar columns to vocabulary tables | `--regconfig`, `--no-create-indexes` | PostgreSQL |
 | `fulltext populate` | Populate sidecar tsvector vectors | `--regconfig` | PostgreSQL |
 | `fulltext drop` | Remove tsvector sidecar columns and indexes | | PostgreSQL |

diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md
@@ -18,22 +18,18 @@ See [Configuration](configuration.md) for the full field reference.
 
 ## Running PostgreSQL tests locally
 
-The test suite includes PostgreSQL-specific tests that skip automatically unless a `test_cdm_db` database is configured in `~/.config/omop/config.toml`. Tests are marked with `@pytest.mark.requires_database("test_cdm_db")` and skipped at collection time when the database is absent — no manual filtering required.
+The test suite includes PostgreSQL-specific tests that skip automatically unless a `test_cdm_db_pg` database is configured in `~/.config/omop/config.toml`. They're resolved via oa-configurator's `isolated_test_database()`, wrapped in this repo's own `pg_db`/`pg_engine`/`pg_session` fixtures, and marked `@pytest.mark.postgresql` (plus `db_dialect` where a test could corrupt shared ORM metadata if run alongside SQLite in the same process). `addopts = "-m 'not db_dialect'"` excludes those by default, so a plain `pytest` run skips them with no manual filtering required. Run them explicitly with `pytest -m postgresql`.
 
-> **This test database is destructive.** The test suite drops and recreates the entire `public`
-> schema on every run. `test_cdm_db` must point to a **dedicated, empty test database**, never
-> to a database that contains real data. The test suite enforces this: it fails loudly (not skips) if the
-> configured database is not marked `test_only = true` in your config.
->
-> Refer to the CI/CD workflows at [cava-devops](https://github.com/AustralianCancerDataNetwork/cava-devops/blob/main/.github/workflows/build-test-postgres.yml) for more details on how integration test runs are typically orchestrated.
+!!! warning "This test database is destructive."
+    `pg_session`-backed tests drop and recreate every non-system schema (not just `public`) both before and after each test. `test_cdm_db_pg` must point to a **dedicated, empty test database**, never to a database that contains real data. The test suite enforces this: it fails loudly if the configured database is not marked `test_only = true` in your config. The suite runs sequentially by design and does not support `pytest-xdist`: it fails loudly under `-n 2` or higher rather than racing another worker's reset.
 
 **Step 1 — Register a test database connection:**
 
 ```bash
 omop-config configure omop_alchemy
 ```
 
-When prompted whether to configure a test database, answer **Y** and supply the connection details for your dedicated test PostgreSQL instance. It will be saved as `test_cdm_db` with `test_only = true`.
+When prompted whether to configure a test database, answer **Y** and supply the connection details for your dedicated test PostgreSQL instance. It will be saved as `test_cdm_db_pg` with `test_only = true`.
 
 > **Note on permissions**: the test suite disables FK constraint triggers during bulk vocabulary
 > loads, an operation PostgreSQL restricts to superusers. Ensure the test database user has
@@ -45,4 +41,4 @@ When prompted whether to configure a test database, answer **Y** and supply the
 pytest -v tests/
 ```
 
-PostgreSQL tests auto-skip when `test_cdm_db` is not configured; all other tests run regardless.
+PostgreSQL tests are excluded from a plain `pytest` run by default (see above); run `pytest -m postgresql` to include them, or `pytest -v tests/ -m postgresql` for verbose output. They still auto-skip if `test_cdm_db_pg` is not configured.
diff --git a/docs/models/clinical/observation.md b/docs/models/clinical/observation.md
@@ -1,4 +1,4 @@
-# oubservation
+# observation
 
 > Documentation coming soon.
 

diff --git a/docs/models/index.md b/docs/models/index.md
@@ -43,3 +43,34 @@ validation, and reuse.
 - [Drug Era](derived/drug_era.md)
 - [Dose Era](derived/dose_era.md)
 - [Cohort & Cohort Definitions](derived/cohort.md)
+
+---
+
+## Health Economic
+
+- [Cost](health_economic/cost.md)
+- [Payer Plan Period](health_economic/payer_plan_period.md)
+
+---
+
+## Metadata
+
+- [CDM Source](metadata/cdm_source.md)
+- [Metadata](metadata/metadata.md)
+
+---
+
+## Structural
+
+- [Episode](structural/episode.md)
+- [Episode Event](structural/episode_event.md)
+- [Fact Relationship](structural/fact_relationship.md)
+
+---
+
+## Unstructured
+
+- [Note](unstructured/note.md)
+- [Note NLP](unstructured/note_nlp.md)
+- [Image](unstructured/image.md)
+- [Image Feature](unstructured/image_feature.md)
diff --git a/docs/models/vocabulary/index.md b/docs/models/vocabulary/index.md
@@ -1,4 +1,4 @@
-# vocaubulary Models
+# vocabulary Models
 
 This section contains ORM models corresponding to OMOP CDM
 vocabulary tables.
Original file line number	Diff line number	Diff line change
Expand Up		@@ -17,3 +17,4 @@ the immutability and interpretability of the underlying CDM tables.

		- [Backend Compatibility](backends.md)
		- [PostgreSQL Full-Text Search](fulltext.md)
		- [Vocabulary Load Performance](vocabulary_load_performance.md)