RBPNet preliminary implementation + new Saluki ablation analysis

README.md

@@ -4,18 +4,27 @@ TranscriptML is a toolkit for training, evaluating, and interpreting RNA
 sequence-to-function models. It provides command-line tools and reusable Python
 APIs for preparing sequence datasets, training models, evaluating held-out
 predictions, and investigating learned sequence features with analyses such as
-in silico mutagenesis, motif ablation, context scans, etc.
+in silico mutagenesis, transcript-region and exon-junction ablation, motif
+ablation, context scans, etc.
 
-TranscriptML currently supports two main workflows:
+TranscriptML currently supports three main workflows:
 
 - **Saluki** predicts transcriptome-wide RNA stability from transcript sequence,
   coding-frame annotations, and splice sites.
 - **MPRA-LegNet** models MPRA measurements from variable sequence inserts and
   supports targets such as RNA stability, translation, protein
   output, etc.
+- **RBPNet/eCLIP** converts FASTA/GTF/BAM inputs into a canonical
+  mature-transcript or full-gene coordinate experiment, descriptive windows, explicit selection
+  manifests, and memory-mappable model-ready NumPy bundles, then trains a
+  sequence-only target/control profile model with an optional replicate-aware
+  enrichment likelihood. **This entire workflow is experimental:** it has been
+  minimally tested and has only been confirmed to preprocess data successfully
+  and train reasonable models on PUM2 eCLIP data. It needs substantially more
+  validation than the other TranscriptML workflows.
 
 In the future, I plan to also support [RiboNN](https://www.nature.com/articles/s41587-025-02712-x) modeling of translation efficiency measurements
-and [RBPNet](https://link.springer.com/article/10.1186/s13059-023-03015-7) modeling of RBP binding assays like eCLIP.
+and extend [RBPNet](https://link.springer.com/article/10.1186/s13059-023-03015-7) interpretation and model variants.
 
 ## Installation
 

docs/api.rst

@@ -43,6 +43,56 @@ Sequence controls
    :members: SequenceControlOperation, SequenceControlConfig, normalize_sequence_control_config, apply_sequence_controls_array, apply_sequence_controls_to_bundle
    :member-order: bysource
 
+RBPNet/eCLIP data
+-----------------
+
+.. warning::
+
+   All RBPNet/eCLIP APIs in this section are experimental. Preprocessing and
+   modeling have been minimally tested and have only been confirmed to process
+   data successfully and train reasonable models on PUM2 eCLIP data. They need
+   substantially more validation than other TranscriptML APIs.
+
+.. automodule:: transcriptml.rbpnet.preprocessing
+   :members: Sample, PipelineConfig, preprocess_eclip
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.experiment
+   :members: ProcessedECLIPDataset, TranscriptRecord, SampleRecord, RegionRecord, GenomicBlock
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.windows
+   :members: WindowScanConfig, generate_window_bounds, calculate_gc_fraction, summarize_regions, scan_windows
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.selection
+   :members: SelectionConfig, SelectionManifest, select_regions, load_selection_manifest
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.bundle
+   :members: RBPNetBundleConfig, jitter_crop_offset, make_rbpnet_bundle, load_rbpnet_bundle
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.dataset
+   :members: RBPNetBatch, RBPNetDataset, collate_rbpnet, deduplicate_locus_indices
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.losses
+   :members: RBPNetLossConfig, RBPNetLossOutput, multinomial_nll, replicate_binomial_nll, RBPNetObjective
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.training
+   :members: train_rbpnet_model, evaluate_rbpnet_model, write_rbpnet_predictions
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.evaluation
+   :members: evaluate_rbpnet_report, resolve_rbpnet_checkpoint_indices
+   :member-order: bysource
+
+.. automodule:: transcriptml.rbpnet.evaluation_metrics
+   :members: profile_metrics, enrichment_metrics, replicate_ceiling_metrics, aggregate_observations, calibration_rows, select_representative_examples
+   :member-order: bysource
+
 Models
 ------
 
@@ -66,6 +116,15 @@ Models
    :members: SmallCNNConfig, SmallCNN
    :member-order: bysource
 
+.. warning::
+
+   The RBPNet model API below is experimental and has only received minimal
+   validation on PUM2 eCLIP data.
+
+.. automodule:: transcriptml.models.rbpnet
+   :members: RBPNetConfig, RBPNetOutput, RBPNet, SamePadConv1d, SameLengthConvTranspose1d, theoretical_receptive_field
+   :member-order: bysource
+
 Training and evaluation
 -----------------------
 
@@ -82,7 +141,7 @@ Training and evaluation
    :member-order: bysource
 
 .. automodule:: transcriptml.training.splits
-   :members: random_split_indices, predefined_split_indices, normalize_splits
+   :members: random_split_indices, predefined_split_indices, group_split_indices, validate_group_disjoint, normalize_splits
    :member-order: bysource
 
 .. automodule:: transcriptml.training.metrics
@@ -100,6 +159,10 @@ Interpretation
    :members: ISMResult, compute_ism, max_abs_effect_per_position, save_ism_result
    :member-order: bysource
 
+.. automodule:: transcriptml.interpret.window_ism
+   :members: WindowISMResult, generate_window_starts, compute_window_ism, save_window_ism_result
+   :member-order: bysource
+
 .. automodule:: transcriptml.interpret.codon_ism
    :members: CodonISMResult, compute_codon_ism, mutation_table_writer, save_codon_ism_result
    :member-order: bysource
@@ -108,6 +171,10 @@ Interpretation
    :members: MotifAblationResult, motif_ablation, save_motif_ablation_result
    :member-order: bysource
 
+.. automodule:: transcriptml.interpret.region_ablation
+   :members: RegionAblationConfig, RegionAblationInstance, RegionAblationResult, region_ablation, save_region_ablation_result
+   :member-order: bysource
+
 .. automodule:: transcriptml.interpret.context
    :members: MotifContextResult, motif_context_scan, save_motif_context_result
    :member-order: bysource
@@ -130,6 +197,10 @@ Plotting
 Run setup
 ---------
 
+.. automodule:: transcriptml.workflows.chromosome_cv
+   :members: ChromosomeCVPlan, ChromosomeCVResolution, create_chromosome_cv_plan, save_chromosome_cv_plan, load_chromosome_cv_plan, resolve_chromosome_cv_plan
+   :member-order: bysource
+
 .. automodule:: transcriptml.workflows.init_run
    :members: init_run
    :member-order: bysource

docs/conf.py

@@ -47,7 +47,7 @@
 
 # Useful autodoc behavior
 autosummary_generate = True
-autodoc_mock_imports = ["torch", "typing_extensions"]
+autodoc_mock_imports = ["torch", "typing_extensions", "h5py", "pyarrow", "pysam", "scipy"]
 autodoc_typehints = "description"
 napoleon_google_docstring = True
 napoleon_numpy_docstring = True

docs/index.rst

@@ -11,12 +11,21 @@ two common starting points:
   with LegNet.
 
 Interpretation tools include single-nucleotide in silico mutagenesis (ISM),
-motif ablations, motif context scans, motif epistasis analyses, and
+region/junction ablations, motif ablations, motif context scans, motif epistasis analyses, and
 Saluki-specific codon ISM. These analyses can expose learned regulatory
 sequence features as well as technical artifacts in the model or assay.
 
-RiboNN support for translation measurements and RBPNet support for RBP binding
-measurements are planned but not yet implemented.
+RBPNet/eCLIP preprocessing, descriptive scanning, region selection,
+materialized dataset construction, and structured profile/enrichment training
+are supported.
+
+.. warning::
+
+   The entire RBPNet/eCLIP workflow is experimental, including preprocessing,
+   scanning and selection, bundle construction, modeling, and evaluation. It
+   has been minimally tested and has only been confirmed to preprocess data
+   successfully and train reasonable models on PUM2 eCLIP data. It needs
+   substantially more validation than other TranscriptML functionality.
 
 Start here
 ----------
@@ -33,5 +42,6 @@ guide describes every Saluki, LegNet, and shared training option. Use the
 
    installation
    usage
+   rbpnet
    training_configuration
    api

docs/installation.md

@@ -40,16 +40,24 @@ TranscriptML use.
 
 Optional extras are available for a few heavier workflows:
 
+```{warning}
+The `rbpnet` extra enables an experimental workflow. RBPNet/eCLIP preprocessing
+and modeling have been minimally tested and have only been confirmed to work
+and train reasonable models on PUM2 eCLIP data; they need substantially more
+validation than other TranscriptML functionality.
+```
+
 | Use case | Install |
 | --- | --- |
 | Write codon-ISM tables as Parquet or Arrow | `python -m pip install -e ".[arrow]"` |
 | Summarize and plot codon-ISM tables | `python -m pip install -e ".[analysis]"` |
+| Preprocess eCLIP and build experimental RBPNet datasets | `python -m pip install -e ".[rbpnet]"` |
 | Run the test suite | `python -m pip install -e ".[dev]"` |
 
 Extras can be combined. The `analysis` extra already includes `pyarrow`, so you
 do not need to install both `analysis` and `arrow`. A full analysis and
 development install is:
 
 ```bash
-python -m pip install -e ".[analysis,dev]"
+python -m pip install -e ".[analysis,rbpnet,dev]"
 ```