WIP: Array Views #867
WIP: Array Views #867
Conversation
tiled/catalog/adapter.py
Outdated
| segments = asset.data_uri.lstrip("tiled://").split('/') | ||
| root, *rest = segments | ||
|
|
||
| def deserialize_ndslice(ser_ndslice): |
There was a problem hiding this comment.
TODO: this could be moved to utils or better to a method on NDSlice class.
tiled/client/container.py
Outdated
|
|
||
| from ..structures.array import ArrayStructure | ||
|
|
||
| def serialize_ndslice(ndslice): |
There was a problem hiding this comment.
Same as above; to be moved to utils or an NDSlice method.
tiled/client/container.py
Outdated
| substructures[-1]['chunks'] = out_chunks | ||
| assets.append(Asset(data_uri=f"tiled://{links[0]}", is_directory=False, parameter='data_uri')) | ||
|
|
||
| # TODO: combine substructures if tehre are multiple sub-arrays that are concatenated |
There was a problem hiding this comment.
Here, we can concatenate several subarrays into a single view. Not sure if it's needed or useful, but seems like a sensible generalization.
tiled/catalog/adapter.py
Outdated
|
|
||
| ndslice = deserialize_ndslice(data_source.parameters['slices'][0]) | ||
|
|
||
| adapter = ArrayAdapter.from_array(adapter.read(slice=ndslice)) |
There was a problem hiding this comment.
Possibly use ArrayViewAdapter, although it's probably unnecessary...
|
Capturing notes on discussion:
|
| @@ -52,8 +54,8 @@ def from_array( | |||
| cls, | |||
| array: NDArray[Any], | |||
| *, | |||
| shape: Optional[Tuple[int, ...]] = None, | |||
| chunks: Optional[Tuple[Tuple[int, ...], ...]] = None, | |||
| # shape: Optional[Tuple[int, ...]] = None, | |||
There was a problem hiding this comment.
we never use these...🤔
| @@ -139,7 +163,7 @@ def read_block( | |||
| """ | |||
| # Slice the whole array to get this block. | |||
| slice_, _ = slice_and_shape_from_block_and_chunks(block, self._structure.chunks) | |||
| array = self._array[slice_] | |||
| array = self._array[tuple(slice_)] | |||
There was a problem hiding this comment.
_array[...] requires a tuple; even though NDSlice is a subclass of tuple, it doesn't work here. Probably there's a better way to define NDSlice so it can be accepted right away (if we want to go this route at all, of course)
This adds a mechanism for creating views of arrays in Tiled. The source for a view could be either an array or a column of a table already registered in Tiled. the view is represented as its own DataSource with possibly several Assets pointing to a location of the source in the Tiled catalog tree, e.g.
tiled://X/Y/imageortiled://X/Y/table/column_1. There is a possibility to use additional parameters to slice or reshape (TBD?) the original array and present only a subset of its values as a view. Likewise, multiple arrays (declared in separate assets) can be concatenated into a single view (TBD?).Checklist