Skip to content

Handle chunks that cover multiple channels or times (Data Manager 5/9) - #492

Draft
frasercl wants to merge 14 commits into
feature/data-manager-4-sourcesfrom
feature/data-manager-5-keys
Draft

frasercl wants to merge 14 commits into
feature/data-manager-4-sourcesfrom
feature/data-manager-5-keys

Conversation

@frasercl

@frasercl frasercl commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

This PR builds in the assumption that DataManager chunks cover exactly one channel and time point. This leads to a few modifications:

  • ChunkSources must provide a mapping from chunk IDs to "storage keys," where each storage key may cover more than one channel or time point
  • Chunk requests must be tracked separately from individual chunks, since they might not correspond 1-1, and requests may only be cancelled if every chunk in the request has been cancelled.

Also introduces the SlotMap data structure for a little help tracking requests efficiently.

@frasercl
frasercl added this pull request to stack #489 October 5, 2026 22:04
@frasercl
frasercl force-pushed the feature/data-manager-5-keys branch from 05440ce to 2cfba1b Compare October 5, 2026 22:50
@frasercl
frasercl force-pushed the feature/data-manager-5-keys branch from 9a0f288 to d7bd083 Compare October 6, 2026 23:37
@frasercl
frasercl force-pushed the feature/data-manager-5-keys branch from d7bd083 to 2dbeee3 Compare October 7, 2026 21:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant