Support sharding through config and raster_write_kwargs - #1106

Open

melonora wants to merge 52 commits into

scverse:mainfrom

melonora:support_sharding

Commits on Mar 30, 2026

build: change supported dask versions
melonora
committed
7039c00

Commits on Apr 7, 2026

feat[config]: allow for persisting config
melonora
committed
63fbe9a
test: add config fixture
melonora
committed
b7e98b9
test: add tests for config
melonora
committed
558fe97

Commits on Apr 14, 2026

add raster write kwargs to api
melonora
committed
8eebdb0
add tests for raster API
melonora
committed
790be0c
build: add zarrs-python for improved shard io
melonora
committed
1a1c673
CI: change lowerbound dask version
melonora
committed
471f72c
build: correct zarrs
melonora
committed
cd48574
build: change distributed version constraint
melonora
committed
0187fe9
build: support dask and distributed >=2026.3.0
melonora
committed
b629de0
CI: change lowerbound test version of dask
melonora
committed
c2b1375
fix: pre-commit error due to incorrect typehint
melonora
committed
bf5b910
build: include zarrs as dependency
melonora
committed
49441fe
make zarrs codec default
melonora
committed
6334fa8

Commits on Apr 16, 2026

config: change chunks and shards to accomodate for raster and table
melonora
committed
73ca72a
chore: adjust raster_write to new config fields
melonora
committed
c6041bb
docs: add docstring
melonora
committed
278606a
change: add support for providing storage options as list
melonora
committed
f6c0a37
docs: make docstring for raster write kwargs more clear
melonora
committed
36e2271
tests: complete and refactor sharding tests
melonora
committed
50cb6bb
docs: adjust docstring config
melonora
committed
6257096
fix: correct parsing chunks, shards argument
melonora
committed
5525fbf
test: add testing for adjusting chunks with env variable
melonora
committed
44414c1
test: write using settings raster_chunks
melonora
committed
e974647
feat: add raster_write_kwargs to write_element
melonora
committed
bd6249e
test: add test writing multiple elements with raster_kwargs
melonora
committed
2600237

Commits on Apr 27, 2026

fix: handle case of element name other than current element
melonora
committed
bbb4bb6
fix: add contextmanager preventing settings leakage between tests
melonora
committed
b95c710
build: remove sharding dependency group
melonora
committed
03aafc9

Commits on May 12, 2026

Merge branch 'main' into support_sharding
LucaMarconato
committed
f44221a

Commits on May 20, 2026

chore: use JSONdict
melonora
committed
f77cff4
Merge branch 'support_sharding' of github.com:melonora/spatialdata into support_sharding
melonora
committed
d614519
chore: use JSONDict
melonora
committed
10ba46b
docs: add comment passing by create_raster_element_kwargs
melonora
committed
c036d49
docs: consistent arrow format and comment renaming chunks/shards
melonora
committed
e65c0fb
refactor: create base_options before storage_options
melonora
committed
26cb40b
build: update dependencies
melonora
committed
2642b74

Commits on May 21, 2026

Merge branch 'main' into support_sharding
LucaMarconato
authored
302fd1a
Merge branch 'main' into support_sharding
melonora
authored
1ce8d08

Commits on May 22, 2026

Commits on Jun 26, 2026

Commits on Jun 30, 2026