Adding common unit tests in "gluten-test" or "common" folder instead of adding in each spark version folder. #8599
Unanswered
Surbhi-Vijay
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Currently, "common" folder contains only few helper classes required for test suites. "test" folder contains Gluten specific tests and not necessarily from Spark.
There are test suites which are replicated to all spark version folder having no difference among them.
for example - GlutenDataFrameHintSuite, GlutenCTEInlineSuite etc.
Based on my understanding "common" folder should contain all the common test suites and specific version folder should only contain tests which have any behavior difference in different versions.
After we drop support for "spark-3.2" (ref), there may be many more testcases which can be refactored to a common folder. This will also help in not replicating test classes while upgrading the spark version.
Was there any reason for keeping all testcases for a spark version in a different folder? Does it make sense to refactor testcases to a common folder?
Beta Was this translation helpful? Give feedback.
All reactions