The inspiration for the Challenge came from presentations at the OME Meeting - that demonstrated the level of adoption of OME-Zarr, but highlighted the - poor findability of the data. + that demonstrated the level of adoption of OME-Zarr and the relative ease of + conversion for some community members, but highlighted the often poor findability of the data + and the need for driving the OME-Zarr specification towards version 1.0. +
+ A lofty goal of 1 Petabyte (PB) of data was set, mostly to make clear the scale of what could be.
Work on the challenge started in earnest in July 2024. All progress is tracked in the ome2024-ngff-challenge - repo. It was agreed that the results - of the OME-NGFF Challenge would be presented at the + repository. It was agreed that the deadline for submission would be such that the results + of the OME-NGFF Challenge could be presented at the 2024 Global BioImaging Meeting.
- The Challenge was run via a series of virtual meetings coordinated on the + The Challenge was run via a series of virtual meetings open to all which were coordinated on the Image.sc Forums, with all notes and lists of participants - available. + available. Initial meetings including that at OME2024 focused on defining the scope of + changes that would be made to the OME-Zarr format. A Python-based tool to convert data + to this format was built. Submissions were then collected in simple CSV files with a Zarr + URL per row.
- At the outset, we weren’t sure how much OME-Zarr data was available, if it - could be converted to Zarr V3, or how many organisations would participate. - In the end, we have far exceeded our expectations, with more than 0.5 PByte - of OME-Zarr made available, across a wide range of modalities. + At the outset, it was unclear how much OME-Zarr data was available, if it + could be converted to Zarr V3, and how many organisations would participate. + In the end, we are pleased with the more than 500 Terabytes of data which + have been made available, across a wide range of modalities.
- To make the assembly as accessible as possible, we built the the OME-NGFF - Challenge Viewer [URL?}, which incorporates a CSV with locations of - datasets, and connects datasets to the OME NGFF Validator, for metadata - validation and viewing. + To make the collection as accessible as possible, the + OME-NGFF Challenge Viewer, + parses submissions from contributors and provides a single view across + all Zarr files. Data can be searched by key-word, + filtered and sorted by various metadata and browsed with thumbnails, + all generated on the fly. Links allow opening of datasets with the OME NGFF + Validator for metadata validation and viewing. + Please let us know on image.sc or GitHub if you have any issues + or ideas.
- Perhaps the most important outcome of the Challenge is the establishment of - a federated bioimage data system based on OME-Zarr. To our knowledge, this - is the largest federated bioimage data collection ever assembled. + Looking back, perhaps the most rewarding outcome of the Challenge is that with a remarkably + modest investment of time and cloud resources, we have almost inadvertently prototyped + a federated bioimage data system based on OME-Zarr, the largest one we know of. + We hope future challenges will continue to push the state-of-the-art forward.
diff --git a/ome2024-ngff-challenge/src/Main.svelte b/ome2024-ngff-challenge/src/Main.svelte index b4cc5b5..83a8d7f 100644 --- a/ome2024-ngff-challenge/src/Main.svelte +++ b/ome2024-ngff-challenge/src/Main.svelte @@ -173,9 +173,10 @@