-
Notifications
You must be signed in to change notification settings - Fork 576
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add fuzzer based on honggfuzz #312
Conversation
Signed-off-by: Chojan Shang <[email protected]>
Signed-off-by: Chojan Shang <[email protected]>
Hi, @Dandandan @andygrove With this patch, we can easily to run the fuzz tool for sqlparser-rs and find more fuzz issues, great for robustness. |
Pull Request Test Coverage Report for Build 901777498
💛 - Coveralls |
Signed-off-by: Chojan Shang <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the delay in review. I am trying to pitch in and help out now to clear the backlog.
This is very cool @PsiACE and @BohuTANG -- thank you. It is cool that it seems you have found a bug with this fuzzer already.
My biggest question is about the input -- does the fuzzer simply check against random inputs or does it somehow know how to randomly make sql statements?
@@ -2358,8 +2358,7 @@ impl<'a> Parser<'a> { | |||
]) // This couldn't possibly be a bad idea | |||
})? | |||
.into_iter() | |||
.filter(|i| i.is_some()) | |||
.map(|i| i.unwrap()) | |||
.flatten() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it correct that this is a real bug that was found/fixed with the fuzzer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no, just make clippy happy
|
||
fn main() { | ||
loop { | ||
fuzz!(|data: String| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is pretty cool. Where does the data come from? Is it just random strings? Or does this somehow know how to make valid sql statements?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see some related comments here: #211 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now it is some random bytes, but can be used to find bugs. Optimization can be continued later, refer to: https://www.cockroachlabs.com/blog/sqlsmith-randomized-sql-testing/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In a past project, we had a harness that could generate random SQL queries and it found many bugs -- such tests are a wonderful way to help database software mature
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filed apache/datafusion#913 for DataFusion. Thanks for the pointer @PsiACE
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is cool and having initial fuzzing, even if just random strings to start, is a good first step.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is pretty cool. Looking forward to the day when we could have sql syntax specific fuzzer :)
Also see: #211