MetricResolutionError: year 0 is out of range #7712
Unanswered
MMoussabbih
asked this question in
Support
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello everyone,
I'm trying to profile a dataset using an UserConfigurableProfiler, but I'm getting this error :
Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/great_expectations/execution_engine/execution_engine.py", line 626, in _process_direct_and_bundled_metric_computation_configurations metric_fn_bundle=metric_fn_bundle_configurations File "/usr/local/lib/python3.7/site-packages/great_expectations/execution_engine/sparkdf_execution_engine.py", line 725, in resolve_metric_bundle res = df.agg(*aggregate["column_aggregates"]).collect() File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/sql/dataframe.py", line 819, in collect File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/serializers.py", line 151, in load_stream File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/serializers.py", line 173, in _read_with_length File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/serializers.py", line 452, in loads File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/sql/types.py", line 1729, in <lambda> File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/sql/types.py", line 823, in fromInternal File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/sql/types.py", line 823, in <listcomp> File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/sql/types.py", line 594, in fromInternal File "/mnt/yarn/usercache/hadoop/appcache/application_1682362944993_0001/container_1682362944993_0001_01_000001/pyspark.zip/pyspark/sql/types.py", line 223, in fromInternal def gi_yieldfrom(self): ValueError: year 0 is out of range
I understood that it's related to a "weird" timestamp value in the dataset, but is there any configuration to disregard this kind of errors ?
I'm using great_expectations==0.15.43
Thanks in advance.
Beta Was this translation helpful? Give feedback.
All reactions