Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
5f6b97a
adding spoqc to spatialaxe
heylf Jul 23, 2026
40bef22
changing spatialdata_wrote.py
heylf Jul 23, 2026
bab69c8
all tests pass now for spoQC
heylf Jul 27, 2026
3254152
updating nextflow configs for the analysis modules
heylf Jul 27, 2026
603637d
update README
heylf Jul 27, 2026
1112d71
adding new module of the final report of spoQC and little changes to …
heylf Jul 31, 2026
374a9b6
change to final_report main.nf and meta.yml
heylf Aug 6, 2026
ac3a289
adding unzip to pipeline for a bigger test data set
heylf Aug 7, 2026
62ccb66
changes to lots tests for spoqc modules main.nf.test
heylf Aug 7, 2026
c5b9e64
adding bigger test data samplesheet_full.csv, test_full.config and ma…
heylf Aug 7, 2026
927aa6e
add untzip to spatialaxe.nf
heylf Aug 7, 2026
5a41ec8
add test config for spoqc subworkflow
heylf Aug 7, 2026
f653407
adding spoqc test, change to .nfignore and spoqc.nf.test and spoqc.nf…
heylf Aug 7, 2026
7e5f020
change to snapshot for final_report main.nf.test.snap
heylf Aug 7, 2026
48339a9
Merge branch 'spoqc' into dev
heylf Aug 7, 2026
a6dfffc
Merge pull request #4 from heylf/dev
heylf Aug 7, 2026
a933a14
change to ro-create-metadata.json
heylf Aug 7, 2026
dcea0c6
change to nextflow_schmea.json
heylf Aug 7, 2026
08217a4
apply prettier modules.json
heylf Aug 7, 2026
d1d824a
changes to modules.json
heylf Aug 7, 2026
9862dc0
changes to nextflow_schmea.json
heylf Aug 7, 2026
73d3207
update nextflow schmema to 2.7.2
heylf Aug 7, 2026
7b88bdd
update subworkflow/nf-core/ultis_nextflow_pipeline
heylf Aug 7, 2026
bb1dba8
tiny change to modules.config
heylf Aug 7, 2026
32ffd20
simplify for spoqc the modules.config
heylf Aug 7, 2026
55d48d9
change to spoqc resources
heylf Aug 7, 2026
0de40f4
add combinatorial cpu labels to base.config
heylf Aug 7, 2026
3df9d01
add combinatorial cpu labels to spoqc modules
heylf Aug 7, 2026
1bc8bf7
bugfix because of new nf-core schema version
heylf Aug 7, 2026
65ff8f2
remove spatialdata write from spoqc subworkflow, hcange ot main.nf
heylf Aug 7, 2026
84fbfb6
allow for qc mode only approach, change to spatialdata_write_meta_erm…
heylf Aug 7, 2026
bb74a0c
add all changes to CHANGELOG.md
heylf Aug 7, 2026
9236b5f
fix in CHANGELOG.md
heylf Aug 7, 2026
ccca85e
change to CHANGELOG.md
heylf Aug 7, 2026
dfecd6e
update to usage.md
heylf Aug 7, 2026
5b52705
add spoqc to output.md
heylf Aug 7, 2026
f9ec855
change to module.jon
heylf Aug 7, 2026
6ff22fd
trim white spaces
heylf Aug 7, 2026
4a14fc1
end-of-file-fixer
heylf Aug 7, 2026
53312b9
end-of-file-fixer
heylf Aug 7, 2026
8a131f5
update nextflow version
heylf Aug 7, 2026
246d2cd
update nextflow version
heylf Aug 7, 2026
14a6a10
upating snapshots
heylf Aug 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/nf-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ jobs:
- isMain: true
profile: "conda"
NXF_VER:
- "25.04.0"
- "26.04.6"
- "latest-everything"
env:
NXF_ANSI_LOG: false
Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,31 @@ Initial release of nf-core/spatialaxe, created with the [nf-core](https://nf-co.

### `Added`

- Added the entire **spoQC tool and subworkflow**: `subworkflows/local/spoqc/main.nf` plus 31 new local modules under `modules/local/spoQC/` (`ambient`, `analysis_category`, `analysis_cluster`, `analysis_overview`, `annotation`, `bubble`, `cell`, `cellcycle`, `combine_masks`, `doublet`, `finalreport`, `general`, `hqcr_celltype`, `hqcr_ident`, `hqpr_bounding_box`, `hqpr_celltype`, `hqpr_clustering`, `hqpr_metrices`, `hqpr_refinement`, `hqtr_ac`, `hqtr_bounding_box`, `hqtr_celltype`, `hqtr_clustering`, `hqtr_metrices`, `hqtr_qv`, `hqtr_refinement`, `marker`, `model`, `transcript`, `void`, `whole_slide`), each with its own `main.nf`, `meta.yml`, and nf-test suite (`tests/main.nf.test` + snapshot).
- Wired spoQC into `workflows/spatialaxe.nf`.
- Added `bin/spatialdata_write.py` support for spoQC's SpatialData output.
- New test configs: `conf/tests/test_spoqc.config` and `conf/tests/test_full_spoqc.config`
- New samplesheet for a full test: `assets/samplesheet_full.csv` with `conf/test_full.config`.
- `conf/base.config`: adding combinatorial label system — separate `process_{tiny,low,mid,high,xl}_cpus`, `_mem`, and `_time` labels, each scaling with `task.attempt`.
- Added `nf-core/unzip` module usage so the pipeline can unpack the larger test dataset.
- samplesheet redefinition: `sample,bundle,image,annotation,stainings`, samplesheet allows for two additional optional columns `annotation,stainings` that are useful for the QC subworkflow.
- `spatialdata_write_meta_merge/main.nf`: Change to subworkflow to account for proper `qc` mode.
- Change to `bin/spatialdata_write.py`: Adding an `all` mode to set all available features to `True`, which is important for QC.

### `Fixed`

- Change to input validation in `spatialaxe.nf`: Moving `morphology_focus/` folder into `bundle_optional_files` because not all Xenium bundles have such a folder.

### `Dependencies`

### `Deprecated`

- `nextflow.config`: bumped the `nf-schema` plugin version.
- `subworkflows/nf-core/utils_nfschema_plugin/main.nf`: added a new `cli_typecast` input (pass `null` to keep default behavior), renamed the `parametersSchema` option key to `parameters_schema` across the help/summary/validate option maps, and fixed how the `--help` text value is resolved.
- `subworkflows/nf-core/utils_nextflow_pipeline/main.nf`: bumped the version.
- `subworkflows/local/utils_nfcore_spatialaxe_pipeline/main.nf`: passes the new `cli_typecast` argument (`null`) through to `UTILS_NFSCHEMA_PLUGIN`.
- `nextflow_schema.json`: cleanup driven by the new schema version.

## 1.0.1 - [06.08.2026]

Hotfix to tackle some bugs
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
[![GitHub Actions Linting Status](https://github.com/nf-core/spatialaxe/actions/workflows/linting.yml/badge.svg)](https://github.com/nf-core/spatialaxe/actions/workflows/linting.yml)[![AWS CI](https://img.shields.io/badge/CI%20tests-full%20size-FF9900?labelColor=000000&logo=Amazon%20AWS)](https://nf-co.re/spatialaxe/results)[![Cite with Zenodo](http://img.shields.io/badge/DOI-10.5281/zenodo.20733817-1073c8?labelColor=000000)](https://doi.org/10.5281/zenodo.20733817)
[![nf-test](https://img.shields.io/badge/unit_tests-nf--test-337ab7.svg)](https://www.nf-test.com)

[![Nextflow](https://img.shields.io/badge/version-%E2%89%A525.04.0-green?style=flat&logo=nextflow&logoColor=white&color=%230DC09D&link=https%3A%2F%2Fnextflow.io)](https://www.nextflow.io/)
[![Nextflow](https://img.shields.io/badge/version-%E2%89%A526.04.6-green?style=flat&logo=nextflow&logoColor=white&color=%230DC09D&link=https%3A%2F%2Fnextflow.io)](https://www.nextflow.io/)
[![nf-core template version](https://img.shields.io/badge/nf--core_template-3.4.1-green?style=flat&logo=nfcore&logoColor=white&color=%2324B064&link=https%3A%2F%2Fnf-co.re)](https://github.com/nf-core/tools/releases/tag/3.4.1)
[![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/)
[![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/)
Expand Down Expand Up @@ -48,6 +48,7 @@ The pipeline supports the following tools:
- QC methods:
- [MultiQC Xenium Extra Plugin](https://github.com/MultiQC/xenium-extra)
- [OPT](https://github.com/JEFworks-Lab/off-target-probe-tracker)
- [spoQC](https://github.com/heylf/spoQC)

## Usage

Expand Down
4 changes: 2 additions & 2 deletions assets/samplesheet.csv
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
sample,bundle,image
test_run,https://raw.githubusercontent.com/nf-core/test-datasets/spatialaxe/xenium_bundle.tar.gz,
sample,bundle,image,annotation,stainings
test_run,https://raw.githubusercontent.com/nf-core/test-datasets/spatialaxe/xenium_bundle.tar.gz,,,0;
2 changes: 2 additions & 0 deletions assets/samplesheet_full.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
sample,bundle,image,annotation,stainings
test_run,https://raw.githubusercontent.com/nf-core/test-datasets/spatialxe/Xenium_V1_Protein_Human_Kidney_tiny_outs.zip,,,0;1
10 changes: 10 additions & 0 deletions assets/schema_input.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,16 @@
"type": "string",
"pattern": "^\\S+$",
"errorMessage": "You can provide an image. If you do not then please leave the field empty."
},
"annotation": {
"type": "string",
"pattern": "^\\S+$",
"errorMessage": "You can provide an annotation. If you do not then please leave the field empty."
},
"stainings": {
"type": "string",
"pattern": "^\\S+$",
"errorMessage": "The indices of the image channels of the individual stainings you want to analyse."
}
},
"required": ["sample", "bundle"]
Expand Down
11 changes: 11 additions & 0 deletions bin/spatialdata_write.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,17 @@ def main():
nucleus_labels = False
cells_as_circles = False

# set sd variables based on the coordinate space
if args.coordinate_space == "all":
cells_labels = True
nucleus_labels = True
cells_boundaries = True
nucleus_boundaries = True
cells_as_circles = True

print(f"[NOTE] Apply cells_boundaries {cells_boundaries}")
print(f"[NOTE] Apply nucleus_boundaries {nucleus_boundaries}")

if args.format == "xenium":
sd_xenium_obj = xenium(
args.bundle,
Expand Down
90 changes: 90 additions & 0 deletions conf/base.config
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,82 @@ process {
maxRetries = 2
}

// =========================================================================
// Combinatorial CPU labels
// =========================================================================

// =========================
// CPU
// =========================

withLabel:process_tiny_cpu {
cpus = { 1 * task.attempt }
}

withLabel:process_low_cpus {
cpus = { 4 * task.attempt }
}

withLabel:process_mid_cpus {
cpus = { 8 * task.attempt }
}

withLabel:process_high_cpus {
cpus = { 12 * task.attempt }
}

withLabel:process_xl_cpus {
cpus = { 30 * task.attempt }
}

// =========================
// Memory
// =========================

withLabel:process_tiny_mem {
memory = { 10.GB * task.attempt }
}

withLabel:process_low_mem {
memory = { 30.GB * task.attempt }
}

withLabel:process_mid_mem {
memory = { 50.GB * task.attempt }
}

withLabel:process_high_mem {
memory = { 100.GB * task.attempt }
}

withLabel:process_xl_mem {
memory = { 250.GB * task.attempt }
}

// =========================
// Time
// =========================

withLabel:process_tiny_time {
time = { 4.h * task.attempt }
}

withLabel:process_low_time {
time = { 8.h * task.attempt }
}

withLabel:process_mid_time {
time = { 16.h * task.attempt }
}

withLabel:process_high_time {
time = { 24.h * task.attempt }
}

withLabel:process_xl_time {
time = { 72.h * task.attempt }
}

// =========================================================================
// GPU labels
// =========================================================================
Expand All @@ -100,4 +176,18 @@ process {
time = { 16.h * task.attempt }
}

// =========================================================================
// Extra labels
// =========================================================================

// That needs to be included for some processes, esle kalleido failes because of tmp dir issues.
withLabel:spoqc {
beforeScript = {
"""
export TMPDIR=\${SLURM_TMPDIR:-\$NXF_TASK_WORKDIR/tmp}
mkdir -p \$TMPDIR
"""
}
}

}
9 changes: 9 additions & 0 deletions conf/modules.config
Original file line number Diff line number Diff line change
Expand Up @@ -378,4 +378,13 @@ process {
mode: params.publish_dir_mode,
]
}

// ---------------------------- spoQC -----------------------------------------

withName: '.*SPOQC_.*' {
publishDir = [
path: { "${params.outdir}/spoqc" },
mode: params.publish_dir_mode,
]
}
}
3 changes: 2 additions & 1 deletion conf/test_full.config
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ params {
config_profile_description = 'Full test dataset to check pipeline function'

// Input data
input = "${projectDir}/assets/samplesheet.csv"
input = "${projectDir}/assets/samplesheet_full.csv"
outdir = 'results'
mode = 'coordinate'
}
35 changes: 35 additions & 0 deletions conf/tests/test_full_spoqc.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for running minimal tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Defines input files and everything required to run a fast and simple pipeline test.

Use as follows:
nextflow run nf-core/spatialaxe -profile test,<docker/singularity> --mode <MODE> --outdir <OUTDIR>

----------------------------------------------------------------------------------------
*/

process {

resourceLimits = [
cpus: 8,
memory: '30.GB',
time: '2.h'
]

withName: UNTAR {
ext.prefix = "test_bundle"
}
}

params {
config_profile_name = 'Test profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Input data
input = "${projectDir}/assets/samplesheet_full.csv"
outdir = 'results'
mode = 'qc'
spoqc = true
}
35 changes: 35 additions & 0 deletions conf/tests/test_spoqc.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for running minimal tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Defines input files and everything required to run a fast and simple pipeline test.

Use as follows:
nextflow run nf-core/spatialaxe -profile test,<docker/singularity> --mode <MODE> --outdir <OUTDIR>

----------------------------------------------------------------------------------------
*/

process {

resourceLimits = [
cpus: 8,
memory: '30.GB',
time: '2.h'
]

withName: UNTAR {
ext.prefix = "test_bundle"
}
}

params {
config_profile_name = 'Test profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Input data
input = "${projectDir}/assets/samplesheet.csv"
outdir = 'results'
mode = 'qc'
spoqc = true
}
50 changes: 50 additions & 0 deletions docs/output.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
- [Xenium Ranger import segmentation](#xenium-ranger-import-segmentation)
- [MultiQC](#multiqc) - Aggregate report describing results and QC from the whole pipeline
- [Pipeline information](#pipeline-information) - Report metrics generated during the workflow execution
- [spoQC](https://github.com/heylf/spoQC) - Report and QC metrics (see [documentation of spoQC](soon))

## Image mode

Expand Down Expand Up @@ -127,6 +128,55 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
- `*.tsv` TSV file containing the gene and transcript information to which each probe aligns
- `stat/`
- `*.tsv` TSV file containing the summary stats
- `spoqc/`
- `report/`
- `annotation/`
- `unsupervised_cell_annotation.tsv` unsupervised cell-type annotation, generated only when no annotation file was supplied as input
- `whole_slide_qc/` whole-slide QC overview report
- `generalqc/` general-purpose QC report
- `bubbleqc/` bubble-artifact QC report
- `doubletqc/` doublet-detection QC report
- `voidqc/` void/empty-region QC report
- `cellqc/` cell-level QC report
- `ambientqc/` ambient-RNA (background gene) QC report
- `hqcr/`
- `hqcr_ident/` high-quality cell region (HQCR) identification report
- `hqcr_celltype/` cell-type-refined HQCR report
- `hqpr/`
- `hqpr_metrices/${staining}/` per-staining high-quality pixel region (HQPR) metrics report
- `hqpr_clustering/${staining}/` per-staining HQPR clustering report
- `hqpr_refinement/${staining}/` per-staining HQPR mask refinement report
- `hqpr_bounding_box/${staining}/` per-staining HQPR bounding-box report
- `hqtr/`
- `hqtr_metrices/` high-quality transcript region (HQTR) metrics report
- `hqtr_ac/` HQTR ambient-contamination probability report
- `hqtr_qv/` HQTR quality-value probability report
- `hqtr_clustering/` HQTR clustering report
- `hqtr_refinement/` HQTR mask refinement report
- `hqtr_bounding_box/` HQTR bounding-box report
- `combine_masks/${staining}/` report combining the HQCR, HQPR and HQTR masks per staining
- `transcriptqc/` transcript-level QC report (against the reference gene panel)
- `cellcycleqc/` cell-cycle scoring QC report
- `modelqc/` model-based QC scoring report
- `analysis/`
- `overview/` downstream QC analysis overview report
- `rna_qc_annotated.h5ad` AnnData object annotated with the combined QC results
- `category/` downstream per-category QC analysis report
- `cluster/` downstream clustering QC analysis report
- `rna_cluster.h5ad` AnnData object with clustering results
- `staining_log.txt` log of the stainings processed by spoQC
- `report.html` final self-contained HTML report aggregating every spoQC step above
- `spoQC_tmp/` intermediate data consumed by later spoQC steps
- `generalqc_output_hqcr.parquet`, `bubbleqc_output_hqcr.parquet`, `doubletqc_output_hqcr.parquet`, `voidqc_output_hqcr.parquet`, `cellqc_output_hqcr.parquet` per-step HQCR contribution scores
- `ambient_output_genes.parquet` ambient RNA gene-signal estimate
- `hqcr_output_mask_raw.parquet` / `hqcr_output_mask_smoothed_raw.parquet` combined raw/smoothed HQCR mask
- `hqcr_output_mask_smoothed_celltype_refined.parquet` cell-type-refined smoothed HQCR mask
- `metrices/hqpr/${staining}/` per-staining HQPR metrics
- `hqpr_${staining}_output_mask_raw/` / `hqpr_${staining}_output_mask_smoothed_raw/` per-staining raw/smoothed HQPR mask
- `metrices/hqtr/` HQTR metrics
- `hqtr_output_ac_prob/` HQTR ambient-contamination probabilities
- `hqtr_output_qv_prob/` HQTR quality-value probabilities
- `hqtr_output_mask_raw/` / `hqtr_output_mask_smoothed_raw/` raw/smoothed HQTR mask
- `multiqc/`
- `multiqc_report.html`: a standalone HTML file that can be viewed in your web browser.
- `multiqc_data/`: directory containing parsed statistics from the different tools used in the pipeline.
Expand Down
12 changes: 12 additions & 0 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,21 @@ nextflow run nf-core/spatialaxe \
--mode qc
```

To run [spoQC](https://github.com/heylf/spoQC) you have to set an additional option:

```bash
nextflow run nf-core/spatialaxe \
-profile <docker/singularity/.../institute> \
--input samplesheet.csv \
--outdir <OUTDIR> \
--mode qc \
--spoqc true
```

- QC methods:
- [MultiQC Xenium Extra Plugin](https://github.com/MultiQC/xenium-extra)
- [OPT](https://github.com/JEFworks-Lab/off-target-probe-tracker)
- [spoQC](https://github.com/heylf/spoQC)

### Image-based Segmentation mode (--mode image): <br>

Expand Down
1 change: 1 addition & 0 deletions main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ workflow NFCORE_SPATIALAXE {
params.stardist_nuclei_model,
params.tiling,
params.xeniumranger_only,
params.spoqc,
)
emit:
multiqc_report = SPATIALAXE.out.multiqc_report // channel: /path/to/multiqc_report.html
Expand Down
Loading
Loading