Skip to content

Commit

Permalink
Comment on handling of HDF5 virtual data sets.
Browse files Browse the repository at this point in the history
  • Loading branch information
danielballan committed Jan 23, 2024
1 parent d8997e8 commit 2ddf620
Showing 1 changed file with 14 additions and 0 deletions.
14 changes: 14 additions & 0 deletions tiled/_tests/test_directory_walker.py
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,20 @@ def read_tiff_with_yaml_metadata(image_uri, metadata_uri, metadata=None, **kwarg

@pytest.mark.asyncio
async def test_hdf5_virtual_datasets(tmpdir):
# A virtual database comprises one master file and N data files. The master
# file must be handed to the Adapter for opening. The data files are not
# handled directly by the Adapter but they still ought to be tracked as
# Assets for purposes of data movement, accounting for data size, etc.
# This is why they are Assets with parameter=NULL/None, Assets not used
# directly by the Adapter.

# One could do one-dataset-per-directory. But like TIFF series in practice
# they are often mixed, so we address that general case and track them at
# the per-file level.

# Contrast this to Zarr, where the files involves are always bundled by
# directory. We track Zarr at the directory level.

layout = h5py.VirtualLayout(shape=(4, 100), dtype="i4")

data_filepaths = []
Expand Down

0 comments on commit 2ddf620

Please sign in to comment.