Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Times get converted to numpy datetime64 instead of Python datetime objects for TabularDataset filtering methods #381

Closed
mx-iao opened this issue Aug 24, 2020 · 1 comment · May be fixed by #391
Closed

Times get converted to numpy datetime64 instead of Python datetime objects for TabularDataset filtering methods #381

mx-iao opened this issue Aug 24, 2020 · 1 comment · May be fixed by #391
Assignees
@diondrapeck
Labels
bug Something isn't working

Comments

@mx-iao
Copy link
Contributor

mx-iao commented Aug 24, 2020 •
edited
Loading

For the filter_dataset_before_time method,

library(azuremlsdk)
ws = get_workspace('workspace-name')
mydate <- as.POSIXct("2011-05-01 17:55:23")
ds <- get_dataset_by_name(ws, 'test_date', version = "latest")
ds <- filter_dataset_before_time(ds, mydate)

the following error gets thrown:

Error in py_call_impl(callable, dots$args, dots$keywords) : AttributeError: 'numpy.ndarray' object has no attribute 'microsecond'

Looks possibly related to this issue, where POSIXct time is getting automatically converted by reticulate to numpy datetim64 objects rather than Python datetime objects, which is what is expected by the underlying Python SDK filter methods for TabularDatasets.

Methods to fix:
@diondrapeck diondrapeck added the bug Something isn't working label Aug 25, 2020
@diondrapeck
Copy link
Contributor

The underlying Python function is expected a datetime.datetime object whereas reticulate converts R date-time objects to either datetime.date or np.datetime64. To bypass this, you can create a native Python datetime.datetime object with reticulate and pass that to filter_dataset_before_time.

Example, for the timestamp 2020-26-08 01:41:30:

datetime <- import("datetime", **convert=FALSE**)

my_date <- datetime$datetime(2020L, 26L, 8L, 1L, 41, 30L)
filtered_ds <- filter_dataset_before_time(ds, my_date)

@diondrapeck diondrapeck mentioned this issue Sep 15, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@diondrapeck diondrapeck

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
2 participants
@diondrapeck @mx-iao