Relax nanosecond datetime restriction in CF time decoding #9618
Conversation
|
Nice, mypy 1.12 is out and breaks our typing, 😭. |
Can we pin it in the CI temporarily? |
Yes, 1.11.2 was the last version. |
kmuehlbauer
force-pushed
the
any-time-resolution-2
branch
from
ca5050d to
f7396cf
Compare
|
This is now ready for a first round of review. I think this is already in a quite usable state. But no rush, this should be thoroughly tested. |
kmuehlbauer
added
topic-CF conventions
topic-cftime
run-upstream
|
Sounds good @kmuehlbauer! I’ll try and take an initial look this weekend. |
…ore/variable.py to use any-precision datetime/timedelta with autmatic inferring of resolution
…ocessing, raise now early
…t resolution, fix code and tests to allow this
No need to rebase/squash manually! As long as this PR can be merged in GitHub, it will be squashed automatically |
There was a problem hiding this comment.
Thanks @kmuehlbauer for updating this—I tried to give it a first thorough look through. Some small suggestions, a few possible minor remaining issues, but I think the big picture is looking good!
Rephrasing and additions to doc string, some test changes. Co-authored-by: Spencer Clark <[email protected]>
for more information, see https://pre-commit.ci
@spencerkclark Thanks very much for the extensive review! I've addressed as much of your comments and suggestions, added todo's to keep track of the remaining. I'll work on your suggestions in time-coding.rst the next days. |
| item = np.timedelta64(getattr(item, "value", item), "ns") | ||
| # from xarray 2025.01.1 xarray allows non-nanosecond resolution | ||
| # so we just convert to_numpy if possible | ||
| if hasattr(item, "to_numpy"): |
There was a problem hiding this comment.
I don't understand timedelta is datetime.timedelta which doesn't have a to_numpy attribute? Do we need a pd.Timedelta clause?
There was a problem hiding this comment.
My indexing knowledge really lacks, so any suggestion would help. This construct was just to get the machinery running again.
| @@ -1165,6 +1266,7 @@ def decode(self, variable: Variable, name: T_Name = None) -> Variable: | |||
|
|
|||
| units = pop_to(attrs, encoding, "units") | |||
| transform = partial(decode_cf_timedelta, units=units) | |||
| # todo: check, if we can relax this one here, too | |||
| @@ -0,0 +1,440 @@ | |||
| .. ipython:: python | |||
There was a problem hiding this comment.
This is great! Thanks for writing this.
Would be good to stick some references to this material in https://xray--9618.org.readthedocs.build/en/9618/user-guide/time-series.html
|
@dcherian Thanks for taking another look here. I'm deep in the weeds to iron out the last remaining issues. Hope to fix this up until next week. 🤞 |
Co-authored-by: Deepak Cherian <[email protected]>
whats-new.rstThis is another attempt to resolve #7493. This goes a step further than #9580.
The idea of this PR is to automatically infer the needed resolutions for decoding/
encodingand only keep the constraints pandas imposes ("s" - lowest resolution, "ns" - highest resolution). There is still the idea of adefault resolution, but this should only take precedence if it doesn't clash with the automatic inference. This can be discussed, though. Update: I've implementedtime-unit-kwarga first try to have default resolutionon decode, which will override the current inferred resolution only to higher resolution (eg.'s'->'ns'). To work towards #4490 the time decoding options (decode_timeanduse_cftimeare bundled withinCFDatetimeCoderwhich is distributed viadecode_timeskwarg.use_cftime-kwarg is deprecated.For sanity checking, and also for my own good, I've created a documentation page on time-coding in the internal dev section. Any suggestions (especially grammar) or ideas for enhancements are much appreciated.
There still might be room for consolidation of functions/methods (mostly in coding/times.py), but I have to leave it alone for some days. I went down that rabbit hole and need to relax, too 😬.
Looking forward to get your insights here, @spencerkclark, @ChrisBarker-NOAA, @pydata/xarray.
Todo:
time_units(where appropriate)CFDatetimeCoderas input fordecode_timeskwarg