Skip to content

Commit

Permalink
Add more creation details
Browse files Browse the repository at this point in the history
  • Loading branch information
will-moore committed Aug 8, 2024
1 parent 3168b93 commit 88ad856
Showing 1 changed file with 32 additions and 5 deletions.
37 changes: 32 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,17 +64,44 @@ Validator available at

Other samples:

- [4496763.zarr](https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/4496763.zarr) Shape `4,25,2048,2048`, from idr0047.
- [9822152.zarr](https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/0.0.5/9822152.zarr) Shape `1,1,1,93184,144384`, from idr0083.
- [4496763.zarr](https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/4496763.zarr) Shape `4,25,2048,2048`, Size `589.81 MB`, from idr0047.
- [9822152.zarr](https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/0.0.5/9822152.zarr) Shape `1,1,1,93184,144384`, Size `21.57 GB`, from idr0083.

<details><summary>details</summary>
<details><summary>Expand for more details on creation of these samples</summary>

Created with ome2024-ngff-challenge commit `f17a6de963`.
<hr>

Took 34 minutes to create:
`4496763.json` was created with ome2024-ngff-challenge commit `0e1809bf3b`.

First the config details were generated with:

```
$ ome2024-ngff-challenge --input-bucket=idr --input-endpoint=https://uk1s3.embassy.ebi.ac.uk --input-anon zarr/v0.4/idr0047A/4496763.zarr params_4496763.json --output-write-details
```

The `params_4496763.json` file was edited to set "shards" to: `[4, 1, sizeY, sizeX]` for each pyramid resolution
to create a single shard for each Z section.

```
# params_4496763.json
[{"shape": [4, 25, 2048, 2048], "chunks": [1, 1, 2048, 2048], "shards": [4, 1, 2048, 2048]}, {"shape": [4, 25, 1024, 1024], "chunks": [1, 1, 1024, 1024], "shards": [4, 1, 1024, 1024]}, {"shape": [4, 25, 512, 512], "chunks": [1, 1, 512, 512], "shards": [4, 1, 512, 512]}, {"shape": [4, 25, 256, 256], "chunks": [1, 1, 256, 256], "shards": [4, 1, 256, 256]}, {"shape": [4, 25, 128, 128], "chunks": [1, 1, 128, 128], "shards": [4, 1, 128, 128]}, {"shape": [4, 25, 64, 64], "chunks": [1, 1, 64, 64], "shards": [4, 1, 64, 64]}]
```

This was then used to run the conversion:

```
ome2024-ngff-challenge --input-bucket=idr --input-endpoint=https://uk1s3.embassy.ebi.ac.uk --input-anon zarr/v0.4/idr0047A/4496763.zarr 4496763.zarr --output-read-details params_4496763.json
```

<hr>

`9822152.zarr` was created with ome2024-ngff-challenge commit `f17a6de963`.

Took 34 minutes to run conversion with this command:
```
$ ome2024-ngff-challenge --input-bucket=idr --input-endpoint=https://uk1s3.embassy.ebi.ac.uk --input-anon zarr/v0.4/idr0083A/9822152.zarr 9822152.zarr --output-shards=1,1,1,4096,4096 --output-chunks=1,1,1,1024,1024 --log debug
```
<hr>
</details>


Expand Down

0 comments on commit 88ad856

Please sign in to comment.