Skip to content

Latest commit

 

History

History
167 lines (139 loc) · 7.07 KB

HEP-0010.md

File metadata and controls

167 lines (139 loc) · 7.07 KB

JPEG2000 File Metadata

  • nickname: JP2K-metadata
  • author(s): Bogdan Nicula
  • contact email: [email protected]
  • date-creation: 2014-07-08
  • date-last-revision: 2014-07-09
  • type: standard
  • status: pending
  • discussion: #1

Abstract

Legacy and lack of full standardisation of solar imaging FITS metadata lead to complicated conditional processing at the time of image display in the Helioviewer system. In some cases, part of the required information is missing from the FITS headers or the data is normalized for different use cases (e.g., constant solar radius). This HEP attempts to solve the problem by shifting this processing at the time of JPEG2000 image preparation. The Helioviewer system clients would have access to a uniform and extensible metadata framework, which may be useful also in other contexts.

Detailed Description

There are two discussions to be had:

  • about the constituents of the metadata
  • about the format of the metadata

Metadata constituents

There are two levels of metadata in the Helioviewer system:

Required

Typical FITS keyword examples in brackets.

  • Observation identification: observatory (TELESCOP), instrument (INSTRUME), detector (DETECTOR), measurement (WAVELNTH)

  • Date of observation start (DATE-OBS in UTC time scale)

  • WCS information about the mapping between pixel space and the observed scene:

    • coordinate frame (WCSNAME, e.g. Helioprojective-cartesian, IVOA HPC)
    • reference point of the coordinate frame (CRVALi in CUNITi units)
    • projection type and axes (CTYPEi, e.g. tangential projection, helioprojective longitude and latitude: HPLN-TAN, HPLT-TAN)
    • number of image pixels (NAXISi)
    • pixel scale (CDELTi in CUNITi units)
    • reference point in the image frame (CRPIXi in pixels)
    • pixel linear transform (CROTAi - deprecated, CDi_j, PCi_j) - this is currently not used in Helioviewer as the images are supposed to have the vertical axis parallel with the solar rotation axis, with solar North on top. Transforming the images would place an undue burden on the helioviewer.org client.

  • Instrument (observer, platform) location: currently DSUN_OBS is required. To enable 3D functionality, the location has to be fully specified. The proposed reference frame is HEEQ (Heliocentric Earth Equatorial), in its spherical representation (DSUN_OBS [m], HGLT_OBS [degree], HGLN_OBS [degree]). The location is geometric (no aberration correction) at DATE-OBS time. This frame has the advantage that, for observers at Earth, HGLN_OBS = 0 and HGLT_OBS = B0 angle, which can be easily computed by the usual software. This is also what is used internally by JHV3D. However, other software using this information may need to incorporate solar rotation computations.

On the last two see also this.

Optional

XMP (proposed format)

There is already a standard format for image metadata: XMP. It is supported in several image formats: PNG, JPEG, JPEG2000, TIFF, thus the metadata can be transfered also across image format conversions. Collages of images may also be supported.

Yada-yada-yada, see Virtual Astronomy Multimedia Project website for the whole sale pitch.

An partially baked example for SWAP (platform location currently missing):

Spectral:
   Band:
      * Ultraviolet
   Notes: Still testing
   CentralWavelength:
      * 17.4
   Bandpass:
      * EUV
   ColorAssignment:
      * Pseudocolor
ResourceURL: http://proba2.oma.be/swap/data/qlk/2014/07/06/2014_07_06__19_24_52__PROBA2_SWAP_SWAP_174.jp2
Title: A very thorough test
Headline: What I said above
ResourceID: 3995611
Temporal:
   IntegrationTime:
      * 10.0
   StartTime:
      * 2014-07-06T19:24:52.021
Instrument:
   * SWAP
Spatial:
   CoordinateFrame: HPC
   ReferencePixel:
      * 512.5
      * 512.5
   Equinox: nan
   ReferenceValue:
      * 0.0
      * 0.0
   ReferenceDimension:
      * 1024
      * 1024
   Scale:
      * 0.000879081694444
      * 0.000879081694444
   CoordsystemProjection: TAN
   Quality: Full
   Rotation: 0.0
Type: Simulation
MetadataDate: 8 July 2014
PublicationID:
   * ads:2013SoPh..286...43S
   * ads:2013SoPh..286...67H
RelatedResources:
   * Testing
   * Python
   * PyAVM
Description: Um, I guess there's not much more to say about this!
Date: 8 July 2014
PublisherID: 125521
Publisher: Bogdan
Rights: Public Domain
ReferenceURL: http://sdac.virtualsolar.org/cgi/search?beats_me_how_to_get_a_rest_query
Credit: Me
MetadataVersion: 1.2
Contact:
   City: Brussels
   Name:
      * Bogdan Nicula
   Country: Belgium
   StateProvince: Flemish Brabant
   Telephone: I think we're getting a little too personal
   Address: None of your business
   PostalCode: What could you possibly need this for?
   Email: [email protected]
ID: 123123123
DatasetID:
   * 12421412
CreatorURL: http://proba2.oma.be
Distance: [3.0]
   Notes: Not much to say, really
Facility:
   * PROBA2
Image:
   ProductQuality: Moderate
Creator: PROBA2 Science Center
Subject:
   Category:
      * A.3.5.3
   Name:
      * Sun EUV

The AVM standard has to be extended, unfortunately, since it's made by Hubble & Spitzer people.

In practice

  • Define the required set of properties and their exact meaning, embracing and extending AVM.
  • This, together with the optional set, will become Helioviewer Metadata v1.0, the version number being part of the spec.
  • When the need arises to extend it, higher versions of metadata spec will be released. The process should be additive, with v1.0 properties always present.
  • Metadata is attached in a JP2 UUID box (0x7575 6964) as per XMP spec. A program will be made available to perform this operation on any JP2 file. IDL IDLffJPEG2000 object can read UUID properties. The JP2 XML box can be removed, as AVM can incorporate the original FITS header (unparsed). Merging files into JPX streams handles JP2 UUID boxes similarly to XML boxes.
  • hv_jp2_verify will be able to verify conformance (file structure, compression settings, metadata structure and contents).

References

Decision Rational

This is a great idea because...