System identification library: classical physics, machine learning, Neural ODE/SDE, and physics-informed hybrid models — all under one unified framework.
This repository provides a comprehensive benchmarking suite for nonlinear system identification combining first-principles physics models with modern deep learning. Models range from classical linear ODEs to Neural SDEs and Mamba sequence models, all trained and evaluated through a single, consistent pipeline on the BAB (Ball-and-Beam) experimental datasets.
- Key Features
- Project Structure
- Installation
- Datasets
- Models
- NN Variants
- Training Pipeline
- Usage
- Output Structure
- Evaluation Metrics
- Visualization
- Notebooks
- License
- 17 model families spanning physics-only, black-box, hybrid, reservoir, stochastic, sequence, and feedforward paradigms.
- 4 NN capacity variants (
compact,base,wide,deep) applicable to all neural models. - Unified training loop — a single
train_model()function handles ODE, SDE, sequence, and feedforward models transparently. - Protocol-2 evaluation — deterministic 50/50 temporal train/test split for core datasets; non-core datasets are test-only.
- Multi-run statistical analysis — train N independent seeds per model spec; aggregate mean/std metrics automatically.
- Rich visualization — prediction overlays, residual plots, raincloud distributions, ACF, spectra, and more.
- Checkpoint & artifact system — save/load model weights, export predictions as
.npz, and persist full run metadata as JSON.
hybrid_modeling/
├── pyproject.toml # Project metadata & dependencies
├── README.md # This file
│
├── bab_datasets/ # Dataset loaders for BAB experiments
│ ├── __init__.py
│ ├── core.py # Dataset registry, loading, preprocessing, velocity estimation
│ └── video.py # Optional video synchronization utilities
│
├── data/
│ ├── sensors/ # Raw .mat experiment files (auto-downloaded if missing)
│ │ ├── 01_rampa_positiva.mat
│ │ ├── 02_rampa_negativa.mat
│ │ ├── 03_random_steps_01..04.mat
│ │ ├── 04_swept_sine.mat
│ │ ├── 05_multisine_01.mat
│ │ └── 06_multisine_02.mat
│ ├── videos/
│ └── labels/
│
├── hybrid_signal_learning/ # Core library
│ ├── __init__.py # Public API re-exports
│ ├── data.py # ExperimentData, Protocol-2 splits, rollout builder
│ ├── io.py # CSV/JSON/NPZ I/O, metric aggregation, run directory management
│ ├── plots.py # All visualization functions
│ ├── train.py # Unified training loop, ODE/SDE/sequence simulation, metrics
│ └── models/ # Model definitions
│ ├── __init__.py # Consolidated public API
│ ├── base.py # InterpNeuralODEBase, NN variants, MLP builder
│ ├── physics.py # Linear ODE, Stribeck ODE
│ ├── blackbox.py # BlackBox, Structured, Adaptive Neural ODEs
│ ├── hybrid.py # Joint & Frozen hybrid models (Linear + Stribeck)
│ ├── esn.py # Continuous-Time Echo State Network
│ ├── ude.py # Universal Differential Equation
│ ├── neural_sde.py # Neural Stochastic Differential Equation
│ ├── sequence.py # GRU, LSTM, TCN, Mamba sequence models
│ ├── feedforward.py # Feedforward NN with lagged I/O features
│ └── factory.py # build_model(), checkpoint save/load, iteration helpers
│
├── scripts/
│ └── run_bab_models.py # Main CLI entry point for training & evaluation
│
└── notebooks/ # Jupyter notebooks for exploration & analysis
├── All_blackboxes.ipynb
├── BAB_models_analysis.ipynb
├── BAB_NODE_models.ipynb
├── HYCO_BAB.ipynb
├── Notebook_with_NODE.ipynb
└── Pedagogical_example_NODE.ipynb
- Python ≥ 3.13
- (Optional) CUDA-enabled GPU for faster training
# Clone the repository
git clone <repository-url>
cd hybrid_modeling
# Create a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate # Linux / macOS
# .venv\Scripts\activate # Windows
# Install the package with all dependencies
pip install -e .
# (Optional) Install development dependencies
pip install -e ".[dev]"| Package | Version | Purpose |
|---|---|---|
torch |
≥ 2.10.0 | Neural network framework |
torchdiffeq |
— | ODE integration (odeint) |
torchsde |
≥ 0.2.6 | SDE integration (sdeint) |
numpy |
≥ 2.4.2 | Numerical computing |
scipy |
≥ 1.16.1 | Signal processing, .mat loading |
matplotlib |
≥ 3.10.8 | Visualization |
scikit-learn |
≥ 1.8.0 | Metrics and preprocessing |
statsmodels |
≥ 0.14.6 | Statistical analysis (ACF, etc.) |
tqdm |
≥ 4.67.2 | Progress bars |
wandb |
≥ 0.24.2 | Experiment tracking |
transformers |
≥ 5.1.0 | Transformer utilities |
optuna |
≥ 4.7.0 | Hyperparameter optimization |
The datasets come from a Ball-and-Beam (BAB) experimental testbed for nonlinear system identification. Each .mat file contains time-series recordings of input voltage (u), output position (y), reference signal (y_ref), and trigger information.
| Dataset Key | File | Description | Protocol-2 Role |
|---|---|---|---|
rampa_positiva |
01_rampa_positiva.mat |
Positive ramp excitation | Test only |
rampa_negativa |
02_rampa_negativa.mat |
Negative ramp excitation | Test only |
random_steps_01 |
03_random_steps_01.mat |
Random step sequence (variant 1) | Train + Test (50/50) |
random_steps_02 |
03_random_steps_02.mat |
Random step sequence (variant 2) | Train + Test (50/50) |
random_steps_03 |
03_random_steps_03.mat |
Random step sequence (variant 3) | Train + Test (50/50) |
random_steps_04 |
03_random_steps_04.mat |
Random step sequence (variant 4) | Train + Test (50/50) |
swept_sine |
04_swept_sine.mat |
Frequency-swept sine wave | Test only |
multisine_05 |
05_multisine_01.mat |
Multi-sine excitation (variant 1) | Train + Test (50/50) |
multisine_06 |
06_multisine_02.mat |
Multi-sine excitation (variant 2) | Train + Test (50/50) |
- Trigger-based cropping — signals are trimmed using the trigger channel to remove idle regions.
- Resampling — default decimation factor of 50 reduces sampling rate for efficiency.
-
Velocity estimation —
$\dot{y}$ is computed via configurable methods:centraldifferences (default), Savitzky–Golay filter, spline derivative, Butterworth filter, or total-variation regularization. -
Auto-download — missing
.matfiles are automatically fetched from the sysid repository intodata/sensors/.
All models predict a 2-D state vector
| # | Model Key | Class | Type | Physics Prior | Neural Component | Description |
|---|---|---|---|---|---|---|
| 1 | linear |
LinearPhysODE |
ODE | Full | None | |
| 2 | stribeck |
StribeckPhysODE |
ODE | Full | None | Linear + Stribeck friction: |
| 3 | blackbox |
BlackBoxODE |
ODE | None | Full | |
| 4 | structured_blackbox |
StructuredBlackBoxODE |
ODE | Kinematic | Partial |
|
| 5 | adaptive_blackbox |
AdaptiveBlackBoxODE |
ODE | Kinematic | Dual NN | Base dynamics NN + near-zero residual NN |
| 6 | ct_esn |
ContinuousTimeESN |
ODE | Kinematic | Reservoir | Echo State Network with augmented reservoir state |
| 7 | hybrid_joint |
HybridJointODE |
ODE | Linear (learnable) | Residual NN | |
| 8 | hybrid_joint_stribeck |
HybridJointStribeckODE |
ODE | Stribeck (learnable) | Residual NN | Stribeck physics + NN residual, jointly trained |
| 9 | hybrid_frozen |
HybridFrozenPhysODE |
ODE | Linear (frozen) | Residual NN | Pre-trained linear params frozen; only NN trains |
| 10 | hybrid_frozen_stribeck |
HybridFrozenStribeckPhysODE |
ODE | Stribeck (frozen) | Residual NN | Pre-trained Stribeck params frozen; only NN trains |
| 11 | ude |
UDEODE |
ODE | Linear SS (learnable) | Residual NN | |
| 12 | neural_sde |
BlackBoxSDE |
SDE | Kinematic | Drift + Diffusion | |
| 13 | gru |
GRUSeqModel |
Sequence | None | Full | GRU encoder → linear decoder |
| 14 | lstm |
LSTMSeqModel |
Sequence | None | Full | LSTM encoder → linear decoder |
| 15 | tcn |
TCNSeqModel |
Sequence | None | Full | Temporal Convolutional Network with causal convolutions |
| 16 | mamba |
MambaSeqModel |
Sequence | None | Full | Selective State Space Model (Mamba architecture) |
| 17 | feedforward_nn |
FeedForwardNN |
Discrete | None | Full | MLP with lagged I/O features, autoregressive rollout |
These models have no learnable neural network component — only physical parameters are optimized.
Classical second-order system:
Parameters: inertia
Extends the linear model with Stribeck friction:
where
These learn dynamics entirely from data using neural networks integrated via torchdiffeq.odeint.
| Model | Kinematic Constraint | Architecture | Notes |
|---|---|---|---|
blackbox |
None | Fully unconstrained | |
structured_blackbox |
Improved sample efficiency | ||
adaptive_blackbox |
Main MLP + zero-init residual MLP | Two-stream acceleration prediction |
These combine a physics backbone with a neural network residual correction:
| Model | Physics Backbone | Params Trainable? | Notes |
|---|---|---|---|
hybrid_joint |
Linear ODE | Yes (jointly) | Physics + NN trained together |
hybrid_joint_stribeck |
Stribeck ODE | Yes (jointly) | Stribeck physics + NN residual |
hybrid_frozen |
Linear ODE | No (frozen) | Pre-trained linear params; only NN trains |
hybrid_frozen_stribeck |
Stribeck ODE | No (frozen) | Pre-trained Stribeck params; only NN trains |
A state-space formulation with learnable linear matrices:
where
Augmented ODE state
The reservoir matrix
Models dynamics with stochastic noise via Ito SDE:
-
Drift
$f$ : structured (kinematic constraint) MLP predicting acceleration. -
Diffusion
$g$ : small MLP producing state-dependent diagonal noise$[\sigma_{\text{pos}}, \sigma_{\text{vel}}]$ . - Integrated with
torchsde.sdeint(Euler–Maruyama method).
These map the full input sequence
| Model | Architecture | Key Feature |
|---|---|---|
gru |
Multi-layer GRU → Linear | Gated recurrent units |
lstm |
Multi-layer LSTM → Linear | Long short-term memory cells |
tcn |
Causal Conv1D residual blocks → Linear | Exponentially increasing dilation for large receptive field |
mamba |
Selective SSM blocks → Linear | Input-dependent state transitions (Mamba architecture) |
A standard MLP operating in discrete time with lagged I/O features:
-
Lag
$L = 10$ (default) — the feature vector has dimension$L \times 3$ . - At inference, predicted outputs are fed back as lagged features for autoregressive free-run simulation.
All neural-network-based models support configurable capacity through NN variants:
| Variant | Hidden Dim | Depth (layers) | Dropout | Typical Use Case |
|---|---|---|---|---|
compact |
64 | 2 | 0.05 | Fast prototyping, limited data |
base |
128 | 3 | 0.05 | Default — balanced performance |
wide |
256 | 3 | 0.05 | Higher capacity, same depth |
deep |
128 | 5 | 0.05 | Deeper representations |
All MLPs use SELU activation with AlphaDropout for self-normalizing behavior.
The training pipeline follows these steps:
┌──────────────────┐ ┌────────────────────┐ ┌──────────────────┐
│ Load Datasets │───▶│ Protocol-2 Split │───▶│ Build Rollout │
│ (BAB .mat) │ │ 50/50 temporal │ │ (concatenate) │
└──────────────────┘ └────────────────────┘ └──────────────────┘
│
┌────────────────────┐ ▼
│ For each model │ ┌──────────────────┐
│ spec × N runs: │◀───│ Compute Valid │
│ │ │ Start Indices │
│ 1. Build model │ └──────────────────┘
│ 2. Train (k-step) │
│ 3. Evaluate all DS │
│ 4. Save checkpoint │
│ 5. Save predictions │
└─────────┬───────────┘
▼
┌────────────────────┐
│ Aggregate metrics │
│ Save CSV / JSON │
│ Plot convergence │
└────────────────────┘
- K-step prediction loss: random windows of
k_steps=20are sampled; models predict forward and MSE is computed against ground truth. - Position-only loss (default): loss targets position channel only, which empirically improves generalization.
- Optimizer: Adam with
lr=0.01. - Batch size: 128 random start indices per epoch.
- Epochs: 1000 (default).
python scripts/run_bab_models.py --quickThis runs a minimal configuration:
- 1 training run (instead of 10)
- 60 epochs (instead of 1000)
- Only the
baseNN variant - Subset of datasets:
multisine_05,multisine_06,random_steps_01,swept_sine
python scripts/run_bab_models.py \
--models linear,stribeck,blackbox,structured_blackbox,hybrid_joint,ude,gru,mamba \
--nn-variants base,wide,deep \
--n-runs 10 \
--epochs 1000 \
--output-root results/bab_runs# Physics-only models
python scripts/run_bab_models.py --models linear,stribeck --n-runs 5
# Only hybrid models
python scripts/run_bab_models.py --models hybrid_joint,hybrid_frozen,ude --nn-variants base,wide
# Only sequence models
python scripts/run_bab_models.py --models gru,lstm,tcn,mamba --nn-variants compact,base| Argument | Default | Description |
|---|---|---|
--output-root |
results/bab_runs |
Base folder for run outputs |
--run-name |
Auto (timestamp) | Custom name for the run folder |
--models |
All 17 models | Comma-separated model keys |
--nn-variants |
base,wide,deep |
Comma-separated NN variant names |
--n-runs |
10 |
Independent training runs per model specification |
--epochs |
1000 |
Number of training epochs |
--lr |
0.01 |
Learning rate (Adam) |
--batch-size |
128 |
Batch size (random start indices per epoch) |
--k-steps |
20 |
Prediction horizon for training loss |
--obs-dim |
2 |
Observation dimension (position + velocity) |
--resample-factor |
50 |
Decimation factor for raw data |
--y-dot-method |
central |
Velocity estimation method (central, savgol, spline, butter) |
--seed |
1234 |
Base random seed |
--device |
auto |
Compute device (auto, cpu, cuda) |
--quick |
false |
Enable smoke-test mode |
--include-datasets |
All | Comma-separated subset of dataset keys |
--save-predictions |
true |
Save full rollout predictions as .npz |
--progress / --no-progress |
true |
Enable/disable tqdm progress bars |
Each run produces a structured output directory:
results/bab_runs/<run_name>/
├── metadata/
│ ├── config.json # Full run configuration
│ ├── best_model_ids_test_r2_pos.json # Best model ID per family (by test R²)
│ └── model_registry.json # Registry of all trained models
├── models/
│ ├── linear__physics__run00.pt # Model checkpoints
│ ├── blackbox__base__run00.pt
│ ├── hybrid_joint__wide__run03.pt
│ └── ...
├── predictions/
│ ├── linear__physics__run00.npz # Full rollout predictions per dataset
│ └── ...
├── tables/
│ ├── training_history.csv # Epoch-by-epoch loss for all models
│ ├── metrics_long.csv # Per-model, per-dataset, per-split metrics
│ └── metrics_aggregate.csv # Mean ± std grouped by model family
└── plots/
└── training_loss_curves.png # Convergence plot for all models
All models are evaluated on both train and test splits with the following metrics:
| Metric | Formula | Description |
|---|---|---|
| RMSE (pos) | Root mean squared error for position | |
| RMSE (vel) | Same, for velocity channel | Root mean squared error for velocity |
| R² (pos) | Coefficient of determination (position) | |
| R² (vel) | Same, for velocity channel | Coefficient of determination (velocity) |
| FIT% (pos) | NRMSE-based fit percentage (position) | |
| FIT% (vel) | Same, for velocity channel | NRMSE-based fit percentage (velocity) |
The library provides a rich set of plotting functions in hybrid_signal_learning/plots.py:
| Function | Description |
|---|---|
plot_training_curves() |
Epoch-vs-loss convergence for all models |
plot_predictions() |
Measured vs. predicted (position + velocity) with train/test split marker |
plot_zoom_position() |
Three zoomed windows (start, middle, end) of position predictions |
plot_residuals() |
Time-series residual plots for position and velocity |
plot_raincloud_models() |
Raincloud (violin + box + scatter) of position residuals per model |
plot_y_vs_yhat() |
Scatter plot of measured vs. predicted position |
plot_acf() |
Autocorrelation function of residuals |
plot_spectra() |
Frequency-domain spectra of residuals |
Each plot function accepts an optional save_path argument to export figures as PNG files at 150 DPI.
| Notebook | Description |
|---|---|
All_blackboxes.ipynb |
Comparison of all black-box model variants |
BAB_models_analysis.ipynb |
Post-hoc analysis of trained models with metrics tables and plots |
BAB_NODE_models.ipynb |
Neural ODE model exploration on BAB data |
HYCO_BAB.ipynb |
Hybrid continuous-time model experiments |
Notebook_with_NODE.ipynb |
Step-by-step Neural ODE workflow |
Pedagogical_example_NODE.ipynb |
Educational introduction to Neural ODEs |
import hybrid_signal_learning as hsl
# Load datasets with Protocol-2 splits
data_map = hsl.load_protocol2_datasets(y_dot_method="central")
# Build a model
model = hsl.build_model("hybrid_joint", nn_variant="wide")
# Build training data
train_data = hsl.build_train_rollout_data(data_map)
tensors = hsl.to_tensor_bundle(t=train_data.t, u=train_data.u, y_sim=train_data.y_sim, device="cpu")
valid_starts = hsl.compute_valid_start_indices(train_data.segments, k_steps=20)
# Train
cfg = hsl.TrainConfig(epochs=500, lr=0.01, k_steps=20)
model, history = hsl.train_model(model=model, tensors=tensors, valid_start_indices=valid_starts, cfg=cfg)
# Evaluate on a dataset
results = hsl.evaluate_model_on_dataset(model=model, ds=data_map["multisine_05"], device="cpu")
print(results["metrics"]["test"])
# Save & reload checkpoint
hsl.save_model_checkpoint("model.pt", model=model, model_key="hybrid_joint", nn_variant="wide", run_idx=0, seed=42)
loaded_model, meta = hsl.load_model_checkpoint("model.pt")| Problem | Solution |
|---|---|
ModuleNotFoundError: No module named 'torch' |
Install PyTorch: pip install torch or follow the official guide. |
| Training is slow on CPU | Set device="cuda" in the config (or leave "auto" with a CUDA-capable GPU). |
RuntimeError: CUDA out of memory |
Reduce batch_size or hidden_size in the model config. |
ImportError: torchcde |
Run pip install torchcde>=0.2.5. |
| Checkpoint fails to load | Ensure the library version matches the one used to save. Use load_model() for automatic class resolution. |
Most video-pipeline defaults now live in `src/config.py` under
`BabVideoPipelineConfig`, with the CLI exposing the most common run and training-mode knobs.
For the built-in BAB datasets:
- label CSVs are auto-resolved from the registry / true-label files,
- frame size defaults to `96x96`,
- additional overrides remain available in `BabVideoPipelineConfig` when needed.
srun -M tinygpu --gres=gpu:1 --time=02:00:00 .venv/bin/python examples/run_pipeline.py \
--dataset swept_sine \
--mode separate \
--ode-model linear_physics \
--encoder-velocity-mode encoder_fd
srun -M tinygpu --gres=gpu:1 --time=02:00:00 .venv/bin/python examples/run_pipeline.py \
--dataset swept_sine \
--mode separate \
--ode-model linear_physics \
--encoder-velocity-mode full_encoder
srun -M tinygpu --gres=gpu:1 --time=02:00:00 .venv/bin/python examples/run_pipeline.py \
--dataset swept_sine \
--mode separate \
--ode-model linear_physics \
--encoder-velocity-mode late_fusion
srun -M tinygpu --gres=gpu:1 --time=02:00:00 .venv/bin/python examples/run_pipeline.py \
--dataset multisine_05 \
--mode separate \
--ode-model linear_physics \
--encoder-velocity-mode full_encoder
srun -M tinygpu --gres=gpu:1 --time=02:00:00 .venv/bin/python examples/run_pipeline.py \
--dataset multisine_05 \
--mode separate \
--ode-model linear_physics
.venv/bin/python examples/run_pipeline.py \
--dataset swept_sine \
--mode enc_ode \
--ode-model linear_physics \
--joint-training-mode subsequence
.venv/bin/python examples/run_pipeline.py \
--dataset swept_sine \
--mode ode_retrain \
--encoder-checkpoint results/swept_sine_separate_windowed_20260312_172115/encoder_best.pt \
--ode-model linear_physics