VectorData Refactor Expandable (#1158)

[mavaylon1]

Motivation

What was the reasoning behind this change? Please explain the changes briefly.
Fix #1158
Fix #1163

Add expandable default for datasets.

How to test the behavior?

Show how to reproduce the new behavior (can be a bug fix or a new feature)

Checklist

• Did you update CHANGELOG.md with your changes?
• Does the PR clearly describe the problem and the solution?
• Have you reviewed our Contributing Guide?
• Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

[codecov]

Codecov Report

❌ Patch coverage is 89.47368% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.78%. Comparing base (42f7311) to head (f6e9d46).

Files with missing lines	Patch %	Lines
src/hdmf/build/objectmapper.py	85.00%	3 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1180      +/-   ##
==========================================
- Coverage   92.86%   92.78%   -0.08%     
==========================================
  Files          41       41              
  Lines        9919     9940      +21     
  Branches     2027     2035       +8     
==========================================
+ Hits         9211     9223      +12     
- Misses        433      437       +4     
- Partials      275      280       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mavaylon1]

@rly This PR is essentially done. I need to go through the pynwb side to make sure what is failing we can ignore through future development. I also want to run the neuroconv tests to make sure those pass as well.

[Copilot]

Pull request overview

This PR implements support for expandable datasets by default and refactors the dimension labels logic to better handle edge cases with varying data shapes and compound dtypes. The changes address issues #1158 and #1163 by:

1. Adding an expandable parameter (defaulting to True) throughout the HDF5 write chain
2. Refactoring __get_dimension_labels_from_spec into __get_spec_info to return both dimension labels and matched spec shapes
3. Using the matched spec shapes to set maxshape when creating datasets, enabling expansion along dimensions defined as None in the spec
4. Properly handling compound dtypes when determining data shapes

Changes:

• Added expandable parameter to write methods (write, write_builder, write_group, write_dataset) with default value True
• Refactored dimension label matching logic in ObjectMapper to also return matched spec shapes
• Added spec_shapes attribute to DatasetBuilder to store matched shapes from specs
• Updated __list_fill__ to use matched spec shapes to set maxshape when expandable=True

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/hdmf/backends/hdf5/h5tools.py	Added expandable parameter to write methods and updated __list_fill__ to set maxshape based on spec shapes
src/hdmf/build/objectmapper.py	Refactored __get_dimension_labels_from_spec to __get_spec_info to return both dimension labels and matched spec shapes; handles compound dtypes correctly
src/hdmf/build/builders.py	Added spec_shapes parameter and property to DatasetBuilder
tests/unit/test_io_hdf5_h5tools.py	Added tests for expandable datasets, reorganized imports, added test for append functionality
tests/unit/helpers/utils.py	Added QuxData and QuxBucket test classes with get_qux_buildmanager helper for testing variable shapes

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 9 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.