Skip to content

Add headless CARTO and EnSiteX importer integration workflow#92

Draft
KestutisMa wants to merge 54 commits into
developfrom
feature/carto-ensite-importer-integration
Draft

Add headless CARTO and EnSiteX importer integration workflow#92
KestutisMa wants to merge 54 commits into
developfrom
feature/carto-ensite-importer-integration

Conversation

@KestutisMa

Copy link
Copy Markdown

Summary

Adds a headless CARTO and EnSiteX conversion workflow suitable for TRE/automated use. The workflow converts mapping exports into OpenEP MAT files, validates the source data and imported OpenEP structures, and writes machine-readable status/progress information plus a human-readable log.

Main Changes

  • Add convert_mapping_case as the machine-facing conversion entry point.
  • Add CARTO ZIP preparation with size-aware extraction to /dev/shm when available, with temp-folder fallback and cleanup.
  • Make CARTO imports batch-testable with explicit map/reference/ECG selections and no GUI prompts.
  • Add header-based EnSiteX export inspection and semantic file-role detection.
  • Add EnSiteX multi-dataset case import, storing bi, uni, and omni datasets in one MAT file.
  • Add select_openep_dataset for selecting a nested OpenEP userdata structure by recording mode.
  • Add validation for mapping inputs, OpenEP userdata, and multi-dataset OpenEP case containers.
  • Add opt-in full-case regression tests for CARTO and EnSiteX.
  • Add TRE integration documentation and a demo-output generation script.

Output Contract

A successful conversion writes:

  • case.mat
  • case.status.json
  • case.log.txt

During conversion it also maintains transient case.progress.json, which is removed after final status publication. If MATLAB is interrupted, a stale progress file remains as evidence of an incomplete job.

Validation

  • Default validation/unit suite via run('run_validation_tests.m').
  • Full CARTO Study1 import regression with RUN_FULL_CARTO_IMPORT_TESTS=1.
  • Full EnSiteX Study3 multi-dataset import regression with RUN_FULL_IMPORTER_SMOKE_TESTS=1.

Full-case tests remain opt-in because they require local clinical exports, large temporary storage, and several minutes to run.

Notes

This is intended as a first integration-ready workflow for TRE/headless conversion. It is not ARO-specific code; ARO can be an initial downstream consumer of the generic conversion contract.

ExternalGuest and others added 30 commits August 30, 2023 22:21
…(v 4.1) where they no longer contain the unipole and bipole mapping channel names. Instead these are accessed from the point XML file within importcarto_mem.
…rent syntax for `matches` for Matlab versions <R2022a
… dummy RF data, a workaround for compatibility with EP workbench. Additionally edited getEarliestActivationSite / getLatestActivationSite to work with files exported from EP Workbench.
Data Export Element continues to have more and more overlapping names, so a separate function to standardize.
importensitex is an adjusted version of importprecision and works with the new files. It needs a helper function to convert into OpenEP format and this can be written separately.
…de naming - both with two unipoles specificed in the bipole electrode naming and without
… full Precision case, if the SOP for exporting the data has been followed
…have a robust routine for comma delimited strings - note this seems to be needed for mapping files but not wave files. It is much slower so we still use regexp for wave files.
@KestutisMa KestutisMa self-assigned this Jul 8, 2026
@KestutisMa
KestutisMa requested a review from drsewilliams July 8, 2026 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants