Skip to content

About

Acute Care Agentic Triage (ACAT): Leakage-Free Clinical Decision Support System via OMOP CDM, LightGBM, and Gemini 2.5 Flash.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

31 Commits

Folders and files

Repository files navigation

Project ACAT: Leakage-Free Cohort Partitioning, Cost-Sensitive TreeSHAP Modeling, and Guideline-Grounded Multi-Agent Decision Support for 30-Day Heart Failure Readmission Risk

Python 3.10+ License: MIT Code Style: Black Clinical Standard: OMOP CDM v5.3 Clinical Guidelines: 2022 AHA/ACC/HFSA

Abstract

  • Research Objective: Investigated methods for developing and evaluating a clinical decision-support prototype for predicting 30-day cardiac readmission risk, interpreting model predictions, and checking generated clinical summaries against heart failure guidelines.

  • Data and Cohort Construction: Used 50,000 encounters from the CMS synthetic patient dataset mapped to OMOP CDM v5.3 through Google BigQuery. Implemented patient-level partitioning with StratifiedGroupKFold, producing 39,986 training encounters and 10,014 holdout audit encounters, with no shared patient IDs reported between the partitions.

  • Predictive Modeling: Trained a class-weighted LightGBM classifier to address outcome imbalance. The repository reports holdout ROC-AUC of 0.9245 and PR-AUC of 0.7327 for the defined 30-day cardiac readmission outcome.

  • Decision-Threshold Analysis: Evaluated classification performance at a threshold of 0.7269, reporting an F1-score of 0.6741, precision of 65.08%, and recall of 69.92%. The threshold-selection procedure should be evaluated separately from final test performance to avoid optimistic estimates.

  • Model Interpretability: Used TreeSHAP to examine feature contributions to individual predictions and identify influential variables, including serum creatinine, ejection fraction, and systolic blood pressure. These analyses describe model behavior rather than establishing causal relationships.

  • Guideline-Informed Decision Support: Implemented a four-stage workflow combining Gemini 2.5 Flash, Pydantic schemas, and ChromaDB retrieval from the 2022 AHA/ACC/HFSA heart failure guidelines. The prototype generates structured summaries and flags potential contraindication-related concerns for human review.

  • Financial Modeling and Monitoring: Developed an HRRP-related financial scenario model using assumed readmission costs, care-coordination reimbursement, outreach costs, and intervention effectiveness. Added Population Stability Index (PSI) calculations to examine changes in clinical feature distributions and illustrate potential monitoring alerts.

  • Limitations and Future Research: The study uses synthetic data and does not establish clinical effectiveness, external generalizability, or realized hospital savings. Future work includes independent clinical validation, calibration assessment, subgroup evaluation, prospective threshold selection, and systematic testing of guideline-related alerts.

  • Research Contribution: Integrates patient-level cohort partitioning, imbalanced classification, model interpretation, guideline retrieval, and distribution monitoring into a reproducible clinical decision-support research prototype.


Research Questions & Hypotheses

Project ACAT is organized around five foundational research questions addressing clinical predictive validity, algorithmic explainability, and multi-agent safety:

  • RQ1 (Leakage-Free OMOP CDM Cohort Engineering): Can longitudinal group-stratified partitioning (StratifiedGroupKFold) eliminate patient identity leakage across repeat acute heart failure encounters within standardized OMOP CDM schemas without sacrificing minority class prevalence distributions?
  • RQ2 (Cost-Utility Threshold Optimization Under Class Imbalance): How does shifting from an arbitrary cutoff ($\tau = 0.50$) to an empirical Precision-Recall optimized operating point ($\tau^* = 0.7269$) under asymmetric class weighting balance emergency triage recall against clinician alarm fatigue?
  • RQ3 (Pathophysiological Fidelity of TreeSHAP Attributions): Do native observation-level TreeSHAP attributions accurately reflect established cardiorenal syndrome mechanics (e.g., non-linear interactions between LVEF, serum creatinine, and systolic blood pressure) without manual heuristic interventions?
  • RQ4 (Guideline-Grounded Agentic Safety Auditing): Can a structured multi-agent architecture coupled with dense vector retrieval over clinical practice guidelines deterministically identify and flag life-threatening drug contraindications (such as ARNi-induced hypotension and MRA-induced hyperkalemia) in real time?
  • RQ5 (Translational Healthcare Economics & Production Drift Governance): How do CMS HRRP penalty structures map to empirical confusion matrices to quantify institutional ROI, and can Population Stability Index (PSI) monitoring establish automated quality-control triggers under seasonal demographic drift?

Data Provenance & Clinical Cohort Engineering

1. Source Database & Canonical Standards

The underlying clinical cohort is derived from the CMS Synthetic Public Dataset mapped to the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM v5.3) hosted on Google Cloud BigQuery:

  • Canonical Schema: bigquery-public-data.cms_synthetic_patient_data_omop
  • Clinical Terminology Standards: Standardized vocabularies adhering to OHDSI specifications, including SNOMED-CT for condition diagnostic codes, RxNorm for medications, and LOINC for laboratory biomarker concepts.
  • Access Protocol: Authenticated access via Google Cloud BigQuery client with analytical queries isolated to the US multi-region.

2. Phenotyping & Leakage-Free Target Definition

  • Index Inpatient Encounter Definition: Primary inpatient admission with a documented heart failure condition occurrence (condition_concept_id $\in {316139, 319835, 437833, 4094876}$).
  • 30-Day Cardiac Readmission Outcome (target_outcome_30d): Unplanned repeat acute cardiovascular admission or post-discharge mortality occurring within a strict 1 to 30-day post-discharge window (DATE_DIFF(nxt.condition_start_date, idx.condition_end_date, DAY) BETWEEN 1 AND 30). Routine outpatient claims, planned follow-ups, and non-cardiac administrative entries are excluded to prevent synthetic label distortion.
  • Lookback Comorbidity Phenotyping: Baseline chronic comorbidities (Type 2 Diabetes mellitus, hypertension, chronic kidney disease) are mapped through ancestral concept sets strictly prior to the index admission date, preventing retrospective target leakage.

3. Cohort Ingestion & Partitioning Pipeline

BigQuery OMOP CDM (CMS Synthetic Cohort)
├── condition_occurrence (Heart Failure SNOMED Codes)
├── person (Demographics, Year of Birth, Gender)
└── concept_ancestor (Pre-admission Comorbidity Mappings)
                          │
                          ▼
            [Leakage-Proof Cohort Filter]
  ├── Acute 30-Day Cardiac Readmission Definition
  └── Elimination of Indeterminate Discharges
                          │
                          ▼
             [StratifiedGroupKFold (k=5)]
     Grouped Strictly by person_id (0% Patient Overlap)
     ├── Training Cohort   : 39,986 Encounters (38,516 Patients)
     └── Audit Test Cohort : 10,014 Encounters (9,628 Patients)

4. Group-Stratified Partitioning & Leakage Prevention

In longitudinal electronic health record (EHR) environments, conventional random record-level partitioning (e.g., standard train_test_split) introduces severe patient identity leakage. Because high-risk heart failure patients frequently experience repeated hospital admissions over multi-year trajectories, randomly allocating encounters distributes identical patient biology across both training and evaluation subsets. Under random splitting, tree ensembles memorize patient-specific baseline quirks rather than learning generalized pathophysiological markers of acute decompensation.

To establish true out-of-sample generalizability, Project ACAT enforces strict patient-level boundary isolation using a 5-fold StratifiedGroupKFold architecture grouped strictly on the primary patient identifier (person_id):

  • Grouping Constraint: Encounters sharing the same person_id are deterministically constrained to the same partition, guaranteeing zero patient overlap between training and holdout audit cohorts.
  • Stratification Constraint: Cross-fold partitions preserve the exact real-world clinical minority event prevalence ($\approx 17%$), preventing synthetic prevalence skewing across splits.

Audit Cohort Partitioning Breakdown ($N = 50,000$)

Cohort Partition Total Encounters Unique Patients Repeat Encounter Rate Acute Readmissions (target_outcome_30d = 1) Minority Event Prevalence Cross-Partition Patient Leakage
Training Cohort 39,986 38,516 3.68% 6,988 events 17.48% 0 Patients (Strict Isolation)
Audit Test Cohort 10,014 9,628 3.85% 1,679 events 16.77% 0 Patients (Strict Isolation)
Total Cohort 50,000 48,144 3.71% 8,667 events 17.33% 0.00% Patient Leakage

5. Clinical Feature Data Dictionary

The predictive feature space is derived from longitudinal OMOP CDM v5.3 tables across condition_occurrence, measurement, person, and visit_occurrence. All baseline clinical covariates are windowed strictly prior to or at the index admission to prevent prospective leakage:

Feature Name OMOP Domain / Table Concept Identifier / Standard Data Type Value Range / Units Clinical Rationale & Extraction Window
target_outcome_30d condition_occurrence OMOP Inpatient Lookahead Binary (0/1) ${0, 1}$ Primary Target: Unplanned acute heart failure readmission or death within $[1, 30]$ days post-discharge.
age person OMOP Demographics Integer $18 - 95$ years Inpatient admission age calculated from year_of_birth relative to index visit_start_date.
gender_concept_id person SNOMED / OMOP Standard Categorical ${8507, 8532}$ Biological sex (8507: Male, 8532: Female) mapped from baseline demographic registry.
length_of_stay visit_occurrence OMOP Visit Dimension Integer $1 - 45$ days Total index encounter duration calculated as DATE_DIFF(visit_end_date, visit_start_date, DAY).
ejection_fraction measurement LOINC 88062-5 / SNOMED Continuous $10.0% - 70.0%$ Left Ventricular Ejection Fraction (LVEF); most proximate measurement recorded prior to discharge.
serum_creatinine measurement LOINC 2160-0 Continuous $0.4 - 8.0$ mg/dL Peak serum creatinine recorded during index encounter; surrogate marker of cardiorenal syndrome.
systolic_bp measurement LOINC 8480-6 Continuous $70 - 210$ mmHg Minimum recorded systolic blood pressure prior to discharge; markers below 90 mmHg reflect hypoperfusion.
prior_comorbidity_count condition_occurrence Standard SNOMED Ancestors Integer $0 - 65$ Distinct ancestral chronic condition concepts documented $\ge 1$ day prior to index admission.
diabetes_mellitus condition_occurrence SNOMED 201820 / Ancestors Binary (0/1) ${0, 1}$ History of Type 2 Diabetes Mellitus confirmed via ancestor concept set prior to index visit.

Methodological Contributions & Empirical Findings

1. Cost-Sensitive Predictive Modeling & Decision Threshold Optimization

Clinical datasets exhibiting low acute event prevalence cause standard loss functions to collapse toward the majority class, producing excessive false negatives that prove catastrophic in acute cardiovascular care. Project ACAT configures a gradient-boosted LightGBM ensemble trained under dynamic minority class weighting (scale_pos_weight = 4.7221), neutralizing prevalence imbalance directly during gradient tree construction.

The predictive engine was evaluated on the untouched out-of-sample holdout audit cohort of 10,014 encounters (1,679 confirmed 30-day cardiac readmission events; 16.77% prevalence). Rather than defaulting to an uncalibrated classification threshold ($\tau = 0.50$), an asymmetric utility-optimal operating threshold ($\tau^* = 0.7269$) was derived via empirical Precision-Recall curve harmonic optimization:

Evaluation Metric Baseline / Arbitrary ($\tau = 0.50$) Calibrated ACAT Engine ($\tau^* = 0.7269$) Clinical Significance & Utility
ROC-AUC 0.9245 0.9245 High global discriminative capacity across unseen patient trajectories.
PR-AUC (Average Precision) 0.7327 0.7327 High precision-recall area under clinical class imbalance.
Operating Threshold ($\tau^*$) 0.5000 (Arbitrary) 0.7269 (Utility-Optimal) Threshold tuned to balance alert precision against bedside miss rate.
Minority Precision (Class 1.0) 42.10% 65.08% Protects clinical staff and care coordinators from alarm fatigue.
Minority Recall (Class 1.0) 84.20% 69.92% Correctly flags 7 out of 10 deteriorating patients prior to discharge.
Minority F1-Score (Class 1.0) 0.5612 0.6741 Harmonic balance on 30-day cardiac deterioration events.
Overall Cohort Accuracy 81.30% 88.67% High general stratification accuracy across all inpatient encounters.

Holdout Test Confusion Matrix Breakdown ($N = 10,014$ Encounters)

  • True Positives (TP = 1,174): High-risk patients correctly identified for post-discharge transitional care and GDMT optimization.
  • False Positives (FP = 630): Low-risk patients conservatively flagged, incurring non-detrimental outpatient follow-up calls.
  • False Negatives (FN = 505): Deteriorating patients missed by the decision cutoff, representing unmitigated readmission exposure.
  • True Negatives (TN = 7,705): Hemodynamically stable patients discharged under standard protocols without unnecessary resource utilization.

2. Observation-Level TreeSHAP Explainability

To prevent ungrounded or speculative explanations, Project ACAT integrates observation-level TreeSHAP attributions directly into inference pipelines without manual heuristic overrides. Global feature importance patterns and local patient log-odds decompositions are computed deterministically from the trained tree ensemble.

Bedside Patient Risk Attribution Decomposition (Predicted Prob: 0.019 | Base Log-Odds: -3.944)
├── Ejection Fraction = 38.1%    ──> [-1.52 SHAP Impact] (Protective Preserved Function)
├── Systolic BP = 143 mmHg       ──> [-1.03 SHAP Impact] (Protective Hemodynamic Perfusion)
├── Serum Creatinine = 1.79 mg/dL──> [+0.45 SHAP Impact] (Moderate Cardiorenal Stress Driver)
├── Prior Comorbidities = 44     ──> [-0.38 SHAP Impact] (Chronic Outpatient Baseline)
├── Age = 39                     ──> [+0.30 SHAP Impact] (Younger Baseline Demographic)
├── Length of Stay = 0 days      ──> [-0.07 SHAP Impact] (Brief Inpatient Exposure)
└── Diabetes = 1                 ──> [+0.05 SHAP Impact] (Metabolic Comorbidity Factor)

Global Feature Attribution Rankings (TreeSHAP Beeswarm Analysis)

  • serum_creatinine (Rank 1): Markedly high values (red points) exert strong positive SHAP impacts, identifying cardiorenal syndrome and impaired filtration as the primary driver of readmission risk.
  • ejection_fraction (Rank 2): Severely depressed LVEF values (<35%, blue points) drive log-odds positively toward acute decompensation, whereas preserved LVEF functions as a protective negative driver.
  • prior_comorbidity_count (Rank 3): High multi-morbidity burden incrementally elevates patient frailty and post-discharge vulnerability.
  • systolic_bp (Rank 4): Profound hypotension (<90 mmHg, blue points) drives positive risk attribution, reflecting cardiogenic shock or systemic hypoperfusion.
  • age & Utilization Metrics (Ranks 5–8): Length of stay, biological sex, and diabetes status provide secondary predictive stratification.

3. Guideline-Grounded Multi-Agent Consensus & Bedside Safety Audit

To prevent generative clinical hallucinations and ensure patient safety, Project ACAT coordinates a 4-agent consensus engine driven by Gemini 2.5 Flash and typed Pydantic contracts. The architecture integrates dense vector retrieval over the 2022 AHA/ACC/HFSA Heart Failure Guidelines indexed in ChromaDB using Google's text-embedding-004:

flowchart TD
    A["Unstructured Clinical Progress Note"] --> B["Agent 1: Vitals Normalizer<br/>(Gemini 2.5 Flash + Pydantic Schema)<br/>• Extracts: LVEF, Cr, SBP, Age, Comorbidities"]
    B --> C["Agent 2: Risk Analyst<br/>(LightGBM + Observation TreeSHAP)<br/>• Evaluates Calibrated Risk vs. τ* = 0.7269<br/>• Computes Native Signed SHAP Impacts"]
    C --> D{"Is Predicted Risk >= 0.7269?"}

    D -- "Yes (Critical Risk)" --> E["Agent 3: Guideline Specialist<br/>(ChromaDB RAG - 2022 AHA Guidelines)<br/>• Semantic Search over Heart Failure Guidelines"]
    D -- "No (Standard Risk)" --> F["Standard Clinical Monitoring Pathway<br/>• Routine Inpatient Management"]

    E --> G["Agent 4: Clinical Auditor & Synthesizer<br/>(Gemini 2.5 Flash)<br/>• Reconciles ML Attributions with Guidelines<br/>• Audits Hemodynamic & Renal Contraindications"]
    F --> G

    G --> H["Point-of-Care Gradio Dashboard<br/>• Structured Bedside Brief & Safety Flags"]

    style A fill:#f8f9fa,stroke:#495057,stroke-width:1px
    style B fill:#e7f5ff,stroke:#1c7ed6,stroke-width:1.5px
    style C fill:#fff3bf,stroke:#f59f00,stroke-width:1.5px
    style D fill:#ffe3e3,stroke:#e03131,stroke-width:1.5px
    style E fill:#e6fcf5,stroke:#0ca678,stroke-width:1.5px
    style F fill:#f1f3f5,stroke:#868e96,stroke-width:1px
    style G fill:#f3f0ff,stroke:#7950f2,stroke-width:1.5px
    style H fill:#fff4e6,stroke:#d9480f,stroke-width:1.5px
Loading

Empirical Bedside Safety Audit (Acute Cardiorenal Encounter)

  • Clinical Presentation: 73-year-old male presenting with acute dyspnea, NYHA Class IV symptoms, LVEF 24%, serum creatinine 2.4 mg/dL, and systolic blood pressure 88 mmHg.
  • Agent 1 (Extraction): Extracted structured vital parameters adhering strictly to the PatientVitals schema (age: 73, lvef: 24.0%, creatinine: 2.4 mg/dL, sbp: 88 mmHg).
  • Agent 2 (ML Stratification): Model produced a calibrated probability of 0.9585 (Critical Risk Tier; exceeds $\tau^* = 0.7269$). Top signed TreeSHAP drivers: systolic_bp (+1.4976), serum_creatinine (+1.3507), and prior_comorbidity_count (+1.2371).
  • Agent 3 (Dense RAG Retrieval): Successfully retrieved 2022 AHA/ACC/HFSA guidelines regarding Acute Decompensation Triage, MRA renal safety criteria, and SGLT2i recommendations.
  • Agent 4 (Bedside Contraindication Audit): Reconciled the clinical picture into an actionable brief that intercepted two high-risk clinical errors:
    • Hypotension Safety Flag: Explicitly warned that an SBP of 88 mmHg contraindicates initiation of ARNi (Sacubitril/Valsartan) and acute vasodilators.
    • Renal Hyperkalemia Warning: Flagged that serum creatinine of 2.4 mg/dL requires confirming eGFR > 30 mL/min/1.73m² and potassium < 5.0 mEq/L before administering MRAs (Spironolactone) to prevent lethal hyperkalemia.
    • Guideline-Directed Management: Recommended prioritizing hemodynamic stabilization and IV loop diuretics, with SGLT2 inhibitor initiation once euvolemia is restored.

4. Healthcare Economics (CMS HRRP ROI) & Production Drift Governance

CMS HRRP 30-Day Readmission Financial Impact ($N = 10,014$ Audit Cohort)

To validate real-world operational feasibility, model accuracy was translated into institutional financial impact using established CMS Hospital Readmissions Reduction Program (HRRP) benchmarks:

  • CMS Readmission Cost / Penalty ($C_{\text{readmit}}$): $15,200 per acute cardiac readmission.
  • Care Coordination Reimbursement (CPT 99490 / 99491): $744/year per enrolled high-risk patient.
  • Transitional Nurse Outreach Cost: $650 per flagged encounter.
  • Guideline Medical Optimization Efficacy: 65% reduction in 30-day readmission risk under active post-discharge monitoring.

$$\text{Net Value Preserved} = \text{CDSS Operational Net} - \text{Default Status Quo Exposure}$$

Economic Modeling Parameter Financial Impact Value Operational Formulation & Derivation
Evaluated Audit Population 10,014 encounters Total unseen holdout test cohort.
Status Quo Default Penalty Exposure -$25,520,800.00 All 1,679 events missed ($1,679 \times -$15,200$) under standard care.
True Positive Clinical Value +$11,709,476.00 1,174 TP patients managed ($TP \times [CPT + (C_{\text{readmit}} \times 0.65) - $650]$).
False Positive Outreach Cost -$409,500.00 630 FP patients enrolled in outreach ($630 \times -$650$).
Uncaptured False Negative Penalties -$7,676,000.00 505 unprevented readmission penalties incurred ($505 \times -$15,200$).
CDSS Net Operational Position +$3,623,976.00 Net financial position combining reimbursements, costs, and avoided penalties.
Total Institutional Value Preserved +$29,144,776.00 Net savings and value preserved relative to status quo default exposure.

MLOps Real-Time Population Stability Index (PSI) Drift Monitoring

To guarantee post-deployment reliability and detect clinical covariate shifts in production, real-time background jobs quantify feature drift between baseline training cohorts and incoming inpatient streams using the Population Stability Index (PSI):

$$\text{PSI} = \sum_{b=1}^{B} \left( P_{\text{actual}, b} - P_{\text{expected}, b} \right) \times \ln\left( \frac{P_{\text{actual}, b}}{P_{\text{expected}, b}} \right)$$

where $B$ denotes the number of empirical quantile bins ($B = 10$), and $P_b$ represents the relative sample frequency of encounters falling within bin $b$. In alignment with standard clinical MLOps monitoring protocols:

  • $\text{PSI} &lt; 0.10$ (Stable): Insignificant distributional shift; model calibration is preserved.
  • $0.10 \le \text{PSI} &lt; 0.25$ (Moderate Drift): Noticeable population variation; triggers enhanced clinical audit frequency.
  • $\text{PSI} \ge 0.25$ (Severe Drift): Critical population divergence; triggers automated alerts and dispatches the retraining pipeline.
Clinical Biomarker Evaluated PSI MLOps Drift Status Automated Quality-Control Action
age 0.4681 🚨 RED ALERT (Severe Drift) Automated retraining trigger dispatched; demographic shift detected.
ejection_fraction 0.4079 🚨 RED ALERT (Severe Drift) Influx of advanced heart failure profiles detected.
serum_creatinine 0.9390 🚨 RED ALERT (Severe Drift) Cardiorenal syndrome cluster flagged; recalibration required.
systolic_bp 0.0010 🟢 GREEN (Stable) Hemodynamic blood pressure distribution within acceptable variance.
prior_comorbidity_count 0.0010 🟢 GREEN (Stable) Baseline chronic disease diagnostic prevalence stable.

Repository Structure & Reproducibility Guide

The Project ACAT repository is organized into modular source libraries, serialized clinical artifacts, and a reproducible end-to-end execution notebook designed for auditability:

Project-ACAT/
├── .gitignore                               # Environment, cache, and large artifact exclusions
├── LICENSE                                  # MIT Open-Source Academic License
├── README.md                                # Comprehensive clinical system specification & empirical audit
├── requirements.txt                         # Pinned dependencies for reproducible deployment
│
├── data/
│   ├── raw/                                 # Initial clinical extract directory (immutable)
│   ├── processed/                           # Sanitized cohorts, visual explainability, and governance audits
│   │   ├── train.parquet                    # 39,986 leak-free group-stratified training encounters
│   │   ├── test.parquet                     # 10,014 untouched holdout audit encounters
│   │   ├── shap_summary_plot.png            # Global TreeSHAP feature attribution beeswarm plot
│   │   ├── shap_patient_waterfall.png       # Local patient observation TreeSHAP waterfall decomposition
│   │   ├── financial_roi_summary.json       # CMS HRRP penalty impact and institutional ROI balance
│   │   └── psi_drift_report.json            # Population Stability Index (PSI) drift monitoring audit
│   └── chroma_hf_rag/                       # Vector store directory with persistent ChromaDB index
│
├── guidelines/
│   ├── .gitkeep                             # Directory persistence placeholder
│   └── aha_hf_guidelines_2022.json          # Structured 2022 AHA/ACC/HFSA Heart Failure practice guidelines
│
├── notebooks/
│   ├── .gitkeep                             # Directory persistence placeholder
│   └── 01_ACAT_Acute_Care_Agentic_Triage.ipynb # Complete 6-phase research and demonstration notebook
│
├── src/
│   ├── __init__.py                          # Package initialization marker
│   ├── data_pipeline.py                     # BigQuery OMOP extraction and StratifiedGroupKFold partitioning
│   ├── model.joblib                         # Serialized cost-sensitive LightGBM classifier binary
│   ├── model_metadata.json                  # Model hyperparameter, feature list, and decision threshold registry
│   ├── model_engine.py                      # CardiacRiskEngine: LightGBM inference & native TreeSHAP attribution
│   ├── rag_engine.py                        # ClinicalGuidelineRAG: ChromaDB semantic search with hemodynamic filtering
│   └── agents.py                            # MultiAgentClinicalOrchestrator: 4-agent consensus engine & Gradio UI
│
└── tests/
    └── .gitkeep                             # Unit test suite directory placeholder

Quickstart & Execution Protocol

1. Environment Configuration & Dependencies

Clone the repository and install the validated clinical machine learning and multi-agent libraries:

# Clone the repository
git clone https://github.com/nmemranhussain/Project-ACAT.git
cd Project-ACAT

# Install dependencies in a clean virtual environment
pip install -r requirements.txt

2. Sequential Pipeline Execution Protocol

The pipeline can be audited or reproduced either through the modular src/ modules or sequentially within notebooks/01_ACAT_Acute_Care_Agentic_Triage.ipynb:

  • Phase 1 & 2: OMOP CDM Extraction & Leakage-Free Cohort Partitioning

Authenticates with Google Cloud BigQuery (bigquery-public-data.cms_synthetic_patient_data_omop).Queries primary inpatient heart failure encounters, applies strict 30-day acute readmission lookaheads, and extracts lookback comorbidities without retrospective target contamination.Executes 5-fold StratifiedGroupKFold grouped strictly by person_id, saving train.parquet (39,986 encounters) and test.parquet (10,014 encounters) with guaranteed 0% patient leakage.

  • Phase 3: Cost-Sensitive LightGBM Training & Native TreeSHAP Explainability

Computes dynamic class weighting (scale_pos_weight = 4.7221) and trains the LightGBM classifier.Derives the utility-optimal operating threshold ($\tau^* = 0.7269$) via Precision-Recall curve optimization.Evaluates holdout test performance (ROC-AUC = 0.9245, PR-AUC = 0.7327, $F_1$ = 0.6741).Calculates observation-level TreeSHAP attributions and serializes src/model.joblib, src/model_metadata.json, and visual SHAP plots.

  • Phase 4: Clinical Guidelines Vector Store & Dense RAG Engine

Ingests evidence-based recommendations from guidelines/aha_hf_guidelines_2022.json.Splits and embeds guideline chunks into a local ChromaDB vector store using Google Vertex AI text-embedding-004.Encapsulates semantic search filtered on LVEF, hypotension, and renal contraindications within src/rag_engine.py.

  • Phase 5: Multi-Agent Consensus Orchestration & Gradio Dashboard

Initializes the 4-tier consensus pipeline in src/agents.py using Gemini 2.5 Flash and Pydantic schemas.Executes the full pipeline: Agent 1 (Vitals Normalizer) $\rightarrow$ Agent 2 (Risk Analyst) $\rightarrow$ Agent 3 (Guideline Specialist) $\rightarrow$ Agent 4 (Bedside Clinical Auditor).Launches an interactive Gradio triage interface for real-time unstructured EHR narrative auditing and contraindication cross-checking.

  • Phase 6: Healthcare Economic Modeling & Production Drift Auditing

Computes institutional financial impact under CMS HRRP readmission penalty benchmarks, saving financial_roi_summary.json.Simulates seasonal cardiorenal demographic drift and calculates the Population Stability Index (PSI) across key clinical biomarkers, saving psi_drift_report.json.


Technical & Clinical Limitations

  • Synthetic Cohort Baseline: The empirical training cohort is derived from the CMS Synthetic Public Schema in OMOP CDM v5.3. While this schema provides realistic multi-table relational structures, prospective institutional validation on real-world clinical databases (e.g., MIMIC-IV, eICU-CRD, or health-system EHRs) is essential prior to live clinical deployment.
  • Unmeasured Social Determinants of Health (SDOH): Administrative claims and electronic health records do not fully capture post-discharge socio-environmental factors (medication access, living conditions, transportation, diet, and health literacy) that strongly influence 30-day heart failure readmissions.
  • Continuous Inpatient Trajectory Modeling: The predictive engine aggregates clinical vitals and laboratory biomarkers at the encounter level. It does not model continuous high-frequency intensive care telemetry or intra-admission physiological fluctuations.
  • Decision Support Scope: Project ACAT is designed strictly as an assistive clinical decision support system. It is not an autonomous diagnostic agent. All therapeutic interventions, drug titrations, and hospital discharge orders remain the clinical responsibility of licensed healthcare providers.

AI Disclosure & Academic Integrity

In alignment with best practices for ethical AI in scientific computing and biomedical research:

  • Predictive Determinism: All risk stratification probabilities, decision threshold cutoffs, and feature attribution impacts are calculated deterministically using Scikit-Learn, LightGBM, and TreeSHAP. Generative AI models are strictly prohibited from generating raw numerical risk scores.
  • Constrained Agentic Reasoning: Gemini 2.5 Flash is utilized solely for unstructured clinical note parsing (constrained via typed Pydantic schemas) and bedside clinical audit reconciliation.
  • Development Assistance: Generative AI tools were utilized for code refactoring, test scaffolding, and Markdown documentation formatting.
  • Intellectual Ownership: All experimental designs, leakage-free cohort definitions, SQL queries, cost-utility threshold formulations, TreeSHAP explainability workflows, and economic models were conceptualized, directed, and verified by the author.

Future Research Directions

  1. Subgroup-Specific Fairness & Demographic Thresholding ($\tau_{\text{age}}^*$):
    • Formulate a constrained multiobjective optimization framework (e.g., Pareto-frontier optimization) to derive age-stratified decision cutoffs, mitigating False Negative Rate disparities between younger and geriatric cohorts without exceeding hospital care-management capacities.
  2. Longitudinal State-Space Modeling on EHR Trajectories:
    • Transition from static encounter aggregations to continuous-time structured state-space models (e.g., Mamba / S4) to model irregular temporal trajectories of inpatient vital signs, fluid balance, and biomarker volatility across sequential admissions.
  3. Formal Verification of Agentic Clinical Synthesis:
    • Extend the Clinical Auditor beyond semantic vector retrieval to formal constraint-satisfaction solvers (e.g., SMT / Z3 solvers) to mathematically guarantee that LLM-generated clinical directives strictly satisfy all clinical practice guideline constraints.
  4. Real-Time FHIR Interoperability & CDS Hooks:
    • Implement an asynchronous HL7 FHIR (Fast Healthcare Interoperability Resources) subscription endpoint capable of receiving real-time Encounter/discharged notifications, triggering automated background triage assessments, and returning calibrated risk scores via standard CDS Hooks.

Citation & Attribution

If you utilize the Project ACAT architecture or build upon these findings, please cite the software repository and the clinical guideline:

@misc{hussain2026acat,
  author       = {Hussain, N M Emran},
  title        = {{Project ACAT: Leakage-Free Cohort Partitioning, Cost-Sensitive 
                  TreeSHAP Modeling, and Guideline-Grounded Multi-Agent Decision 
                  Support for 30-Day Heart Failure Readmission Risk}},
  year         = {2026},
  publisher    = {GitHub},
  journal      = {GitHub repository},
  howpublished = {[https://github.com/nmemranhussain/Project-ACAT](https://github.com/nmemranhussain/Project-ACAT)}
}

@article{heidenreich2022aha,
  author       = {Heidenreich, Paul A. and Bozkurt, Biykem and Aguilar, David and 
                  Allen, Larry A. and Byun, Joni J. and Colvin, Monica M. and 
                  Deswal, Anita and Drazner, Mark H. and Dunlay, Shannon M. and 
                  Evers, Linda R. and others},
  title        = {{2022 AHA/ACC/HFSA Guideline for the Management of Heart Failure: 
                  A Report of the American College of Cardiology/American Heart 
                  Association Joint Committee on Clinical Practice Guidelines}},
  journal      = {Circulation},
  volume       = {145},
  number       = {18},
  pages        = {e895--e1032},
  year         = {2022},
  publisher    = {American Heart Association},
  doi          = {10.1161/CIR.0000000000001063}
}

About

Acute Care Agentic Triage (ACAT): Leakage-Free Clinical Decision Support System via OMOP CDM, LightGBM, and Gemini 2.5 Flash.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages