You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the case of integers, columns are returned as int, which is the system word size int, which is probably okay. For smaller width ints on OmniSci, this is ok, since there is no precision loss. Might be unexpected though to have a tinyint column return a 32- or 64-bit int column.
Bool is returned as 1/0, which is conventionally correct, but odd since Python itself has True/False
Modifying these behaviors are necessarily a breaking change, but in the case of losing precision, it seems like the correct thing to do.
The text was updated successfully, but these errors were encountered:
Why is that? 1/10 is not exactly representable as a binary fraction. Almost all machines today (November 2000) use IEEE-754 floating point arithmetic, and almost all platforms map Python floats to IEEE-754 “double precision”. 754 doubles contain 53 bits of precision, so on input the computer strives to convert 0.1 to the closest fraction it can of the form J/2**N where J is an integer containing exactly 53 bits. Rewriting
Update: filed internally as https://jira.omnisci.com/browse/BE-3467 for decimal downcast
When using the execute method to return a list of tuples, the following types lose precision:
double downcast to floatEDIT: pythonfloat
is actually double precision for modern platformsIn the case of integers, columns are returned as
int
, which is the system word size int, which is probably okay. For smaller width ints on OmniSci, this is ok, since there is no precision loss. Might be unexpected though to have a tinyint column return a 32- or 64-bitint
column.Bool is returned as 1/0, which is conventionally correct, but odd since Python itself has
True/False
Modifying these behaviors are necessarily a breaking change, but in the case of losing precision, it seems like the correct thing to do.
The text was updated successfully, but these errors were encountered: