Allow browse files

[lisakaser]

Allow each granule to have a browse file [likely only CNM message needs to change for this issue]



Details:
Data set: use a netcdf, NSIDC-0081DUCk is a good option, in the end browse should work for all science file types
browse: allow all possible options (Research the options that are allowed for browse)
 -> see below
Size summary might need to go into UMM-G? (Amy? -this issue is blocked till Amy returns ) -> see below

Multiple browse per granule are possible
? -> see below

Acceptance criteria:
All browse files are found and end up in the CNM message


UMM-G: confirm that granule size is correctly calculated (confirm if this is with or without browse)


Browse files need to be staged in correct location (same S3 bucket as data and metadata)

[lisakaser]

@afitzgerrell here is the list of questions that currently blocks this issue:

What are all the allowed browse options (file types)?
Does browse file size need to go into UMM-G or is browse file size excluded from size summary in UMM-G?

Are multiple browse per granule an option that we need to allow?

[afitzgerrell]

What are all the allowed browse options (file types)? Based on the MimeTypeEnum listing in Earthdata's UMM-G schema, the allowed options are: “image/jpeg”, “image/png”, “image/gif”, “image/tiff”, & “image/bmp” (although the only ones I'm familiar with seeing for browse images are .jpg, .png, and .tif). Although, something to keep in mind is that any of these are also valid data types for data or ancillary files, either stand alone for a single-file granule + browse, or within a multi-file granule + browse (as noted in the CNM documentation under the Product sub-fields / files description and in the UMM-G documentation at DataGranule, Identifiers section).

Does browse file size need to go into UMM-G or is browse file size excluded from size summary in UMM-G? Browse are discrete files that should not be considered part of a granule's contents, thus should not be included in the UMM-G SizeInBytes value.

Are multiple browse per granule an option that we need to allow? Yes, this would be ideal. Generally, if data producers generate browse images it's typically been one browse per granule. There are data set cases where multiple browse images are included/granule, an example being NISE which has two browse, one for each Hemisphere.

[lisakaser]

Somehow the operator needs to be indicate what are the browse files or where the browse files are. (Either a separate location for browse or a consistent file name ending) -> @afitzgerrell will discuss with Ops to find out their preference

A browse file can not at the same time be a science file as part of the granule.

Browse files will be listed in the CNM message

Browse files will be listed in the UMM-G as URLs but not part of the file size

[afitzgerrell]

I queried OPS this morning and the consensus is: they'd prefer we (ideally) encourage data producers to include "brws" in the file names for browse images, rather than not and requiring the browse images be sequestered together in a different directory from the data files. In a pinch, were we to end up with data sans this keyword and a data producer unable or unwilling to rename files, OPS could add it in.