handling relative/absolute time correctly and consistently

[pavlis]

In an email a few days ago I summarized this problem as follows:

Begin email
Let me explain why this exists - it was not present in my earliest version of this library.d

There are several real life data cases with which we need to be concerned:

1. Top of list happens to data with valid UTC time.

2. Active source data exist in three variants of relative time:

a. Virtually no absolute time reference except what might be in written field notes (oldest data)

b. Approximate time stamps. Most newer active source instruments have a system clock that gives only approximate time. e.g. I have some data from Homestake where the data logger only recorded a time stamp on the segy data files to the nearest 1 s. That is common.

c. Active source data assembled with a true time stamp. Type example are modern long range refraction experiments where the instruments have gps time.

3. Synthetic seismogram. These always have a relative time base to start with but in many applications need to be converted to absolute time to compare to data or time shift the data to match the synthethics (the two are equivalent, of course, but have different data handling implications).

So, the reason for the boolean is to flag cases like 2a and 2b and some situations for 3 where if a shift time is posted it won't be reliable. Those case, in fact, argues for separate getters and setters that I don't think are all their right now._
End email

One thing this email didn't explicitly state but which is a key concept is that data can have time defined as relative but actually be based on utc time. The library handles this with the internal boolean variable "t0_shift_is_valid" in BasicTimeSeries. After the discussion above I actually fixed a bug that had existed in the rtoa code that handle the t0_shift_is_valid logic correctly.

The point for here is that we implemented code to handle storing and retrieving a boolean we used to distinguish cases when the internal time shift variable was reliable or not. I believe that is working right? I think the only residual issue is what the python API should have to make sure we can correctly handle these issues. Ian, can you please clarify what you think we need that may be lacking or what needs to be changes?

[pavlis]

In the process of revising the documentation on timing and writing an expanded document on timing I recognized we MUST add a standardized attribute to the schema. Here is the yaml entry I recommend:

timing_precision:
  type:  double
  concept:  Uncertainty estimate of time stamps (in seconds).  
timing_precision_metric:
  type:  string
  concept:  A keyword describing the metric used to define timing precision.  Use supplied but likely forms are: 'sigma','interquartile', or 'range'.  

These attribute should not be required but needs to be supported as there are many cases where timing precision is a necessary constraint. The second may not be necessary but a minor addition worth putting in for completeness.

[wangyinz]

While it is fine to add optional attributes to the database, I think we do want to keep our default schema as concise as possible. Do you think there is a use case for the two attributes in our current code base? If not, maybe we should just treat it as something a user may add in their own schema file.

[pavlis]

Yes, but even though our code base may not currently use these attributes having them defined as a concept is important.

There are lots of examples like that were we could and perhaps should anticipate attributes people will want. Another example would be moment tensor components that would be essential for anyone working with methods comparing synthethics and data. There are undoubtedly others. We might want to look at the attributes in css3.0 as a starting point. Another example is the ISC's database schema that is an extension of CS3.0. Not at the top of our needs, but something probably worth putting on the to-do list someday.

[wangyinz]

I agree. Actually, in addition to the default schema, we should include a number of extended schemas for different type of research. Our schema can already do that. We will come back to this topic and put together a number of schema that we want to support.

[pavlis]

Roger that. I'd add only one suggestion. This would be an excellent topic for our advisory group - whatever we want to call them.