Skip to content

Add experimental Pyxa (Stellaromics) reader - #425

Draft
ckmah wants to merge 15 commits into
scverse:mainfrom
ckmah:pyxa-reader
Draft

ckmah wants to merge 15 commits into
scverse:mainfrom
ckmah:pyxa-reader

Conversation

@ckmah

@ckmah ckmah commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Adds an experimental reader for Stellaromics Pyxa output, spatialdata_io.experimental.pyxa(), plus a pyxa CLI command.

What it reads

File Element
cell_assigned_gene_v1.csv transcripts (3D points; unassigned transcripts, with a cell_id ending in _-1, are kept and flagged by an assigned column)
cell_by_gene_v1.csv + cell_metadata_v1.csv rna table, with obsm["spatial"] in µm
segmentation_geometries_v1.parquet cell_boundaries (one footprint per cell, annotated by rna) and cell_boundaries_z (one polygon per cell per z-plane)
optional mosaic OME-Zarr (image_path=) mosaic_image, multiscale, using the pyramid levels already in the store

Every element is in µm in the global coordinate system:

  • The segmentation polygons are stored in pixels. The reader scales them to µm, then repairs and checks the ones that the float scaling makes self-intersecting.
  • Their z-plane centre is added as a Z_um column, since shapes are 2D in spatialdata.
  • The xy and z voxel sizes are fitted from a sample of cell_metadata rows, where each cell centroid is given in both pixels and µm. The reader raises an error if the two aren't related by a pure scale.

Each cell's centroid in cell_metadata is exactly the area-weighted centroid of its polygons, which the tests check in x, y and z.

Cells stacked in z have overlapping footprints. cell_boundaries is therefore meant for 2D display and table annotation, not for 2D spatial aggregation.

Specification

There is no public format specification yet, so the reader is under experimental. It is validated against the public Stellaromics/demo dataset (BSD 3-Clause).

Test data

  • xsmall/ in that dataset is a 100 × 100 × 100 µm crop: 187 cells, about 23k transcripts and the DAPI mosaic, about 8 MB in total.
  • prepare_test_data.yaml downloads it to data/pyxa_xsmall.
  • Maintainers: the test data artifact has to be regenerated for CI to pick it up. The workflow notes that this needs a branch in this repository.

Scripts to download and convert the dataset are in giovp/spatialdata-sandbox#65 (pyxa_v1_io).

Tests (tests/test_pyxa.py)

  • Example data: data extent, index integrity of every element (image pixels, transcripts, footprint and per-plane polygons, table values), the table annotation joins, and the CLI with the mosaic.
  • Helpers:
    • column validation;
    • voxel-size fit, including sampling and rejecting metadata that isn't a pure scale;
    • geometry repair;
    • footprint union;
    • multiscale image loading, where a level that was read is distinguishable from one that was recomputed;
    • µm conversion.

Checklist

  • Reader under experimental/; string constants in PyxaKeys
  • Small public test dataset with a permissive license (BSD 3-Clause)
  • Extent and index-integrity proxy tests; tests for the helper functions
  • Visual check of the alignment between image, polygons and transcripts
  • CLI command
  • Download and conversion scripts for spatialdata-sandbox

Narrows the tests/data gitignore rule so this small (~1.5MB) fixture
can be checked in, since no public host exists for Pyxa data (unlike
other readers' CI-downloaded fixtures).
Assembles points (transcripts, dask-backed), shapes (segmentation),
and table (expression + metadata) into one SpatialData object.
Registers pyxa in spatialdata_io.experimental (not the top-level
package) matching the existing precedent for iss, and adds the
`spatialdata-io pyxa` CLI command.
Not part of this repo's build (hatchling-based, not uv) -- it's local
dev tooling artifact from installing test dependencies with uv.
Adds an image_path parameter to pyxa() that loads a full-resolution
OME-Zarr (OME-NGFF v0.5) mosaic image (e.g. DAPI) into
sdata.images["mosaic_image"], using its coordinateTransformations
metadata so it aligns with the points/shapes in microns. The image is
not colocated with the other 4 Pyxa output files in the real pipeline
layout, so it's a separate optional argument rather than discovered
from `path`. Only the full-resolution level (scale0) is read; reusing
the precomputed pyramid is left for later.
After rebasing onto upstream main: import the reader lazily inside
pyxa_wrapper (upstream dropped the globals() injection in __main__),
expose --image-path so CLI options mirror the reader API, and fix
mypy/ruff findings (typed OME-Zarr attrs, zip(strict=True), docstring).
Reader:
- Convert segmentation polygons from pixels to micrometers in the reader,
  so every element is returned in um with an identity transformation.
  Polygons made invalid by the float scaling are repaired with
  shapely.make_valid (structure method, polygonal parts only) and validated.
- Add a Z_um column to the shapes: the z-plane centre, (ZIndex + 0.5) * dz,
  since shapes are 2D in spatialdata. cell_metadata Z_pixels is exactly the
  area-weighted mean ZIndex + 0.5 of each cell.
- Infer the (xy, z) voxel size from a sample of cell_metadata rows and raise
  if pixel and um centroids are not related by a pure scale.
- Make the gene categories known before PointsModel.parse to avoid its
  unknown-categories warning.
- Remove references to internal documentation and pipeline naming.

Tests:
- Replace the checked-in fixture with the xsmall crop (100 um cube) of the
  public Stellaromics/demo Hugging Face dataset, downloaded in
  prepare_test_data.yaml like the other readers' test data.
- Check that area-weighted polygon centroids match the per-cell metadata
  centroids in x, y and z.
- Return two shapes elements: cell_boundaries, one footprint per cell (the
  union of its z-plane polygons, merged in a thread pool) indexed by cell_id
  and annotated by the rna table; and cell_boundaries_z, the per-plane
  polygons with a unique index and cell_id / ZIndex / Z_um columns. A table
  cannot annotate an element with repeated instance ids: the joins either
  failed or duplicated rows.
- Drop the obs index name that clashed with the cell_id column in joins.
- Load every precomputed level of the mosaic OME-Zarr as a multiscale image
  instead of only the full-resolution level.
- Add example-data tests on the xsmall dataset following the contribution
  guide: data extent, index integrity of every element and of the table
  annotation, and the CLI with the mosaic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant