Request: Rule Cleaning for Hashcat.

[Binary-Bear]

0xVavaldi would you please consider the following request?

I found ruleprocessorY.exe because I was looking for a rule cleaning tool. I have collected many rules some of which do not work directly with hashcat or produce non-keyboard characters which I would personally never need. Rule lists containing such rules may waste valuable processing/cracking time.

I understand that ruleprocessorY.exe optimizes all rules and not just rules written for hashcat. I had originally assumed that --hashcat-output would remove all rules that hashcat would otherwise reject but this does not seem to be the case.

As ruleprocessorY.exe is awesome for optimizing rules it seems strange that ruleprocessorY.exe does not remove invalid rules, ie remove any rules that hashcat would reject or which produce a non-keyboard character.

I have made various attempts to find and remove such rules but I am not a programmer. However I have managed to find a number of invalid rules, or rules which produce a non-keyboard characters, and remove them myself. However it would be wonderful if ruleprocessorY.exe had an option to do this properly.

Thank you.

[0xVavaldi]

If you can provide a few examples for me to work with, and example command, and your expected output I will look at it.

[Binary-Bear]

Thank you very much 0xVavaldi.

There are a lot of people looking for a rule cleaning tool so I expect ruleprocessorY.exe will become even more popular.

As hashcat may reject rules that other rule processing tools may accept I think it is important that the user makes it clear which tool they are cleaning the rules for.

Obviously any line that starts with # (containing a comment) or any line which is not a rule should be marked as invalid.

So my first suggestion would be something like...

ruleprocessorY.exe --cleanrules-hashcat -r rules_To_Clean.rule -i invalid.rule -o Cleaned_Rules.rule

--cleanrules-hashcat = "--cleanrules" defines that the user has chosen to "clean" the rules and "-hashcat" defines that the rules are to be cleaned for use with hashcat. The hashcat definition is important because different tools handle different rules.

The above leaves the option for further definitions at a later date such as "--clean-ruleprocessorY" or "--cleanrules-jtr".

-r rules_To_Clean.rule = The input rule list to clean.

-i invalid.rule = The destination file to send all the invalid or rejected rules to. This will allow the user to review what has been removed.

-o Cleaned_Rules.rule = The destination file for the clean (valid) rules.

I am ashamed of my humble efforts but I have had some success filtering the rules by using the following regular expression when piping hashcat through sed.exe.

hashcat.exe -r Original_Rules.rule testpassword.txt --stdout | sed.exe -n "/^Skipping invalid or unsupported rule/p";"/^Cannot convert rule for use on OpenCL/p" | "!sed.exe!" -e "s/^.*on line [0-9]\+\: //">>invalid.rule"

The above outputs the rules which produce errors in hashcat.

I use the following sed.exe expression to catch rules which produce non-keyboard characters.

-n "/\(&#\)[0-9]\{2,\}\(;\)/p"

I have attached a text file containing rules which I have found and removed from a collection of rule files downloaded from various sources.
Invalid_Rules_1.zip

You may notice that ruleprocessorY.exe crashes when trying to output the result of the list to a text file.

ruleprocessorY.exe -w testpass.txt -r Invalid_Rules_1.rule --hashcat-input >"testoutput.txt"

Thank you 0xVavaldi for taking an interest in my request.

[Binary-Bear]

I have found more invalid or poor quality rules within rule files shared on the internet and I have updated the Invalid_Rules.zip file I uploaded earlier.

I have also discovered that sed.exe is not very good at finding extended ASCII characters in Windows so there may be more.

Invalid_Rules_2.zip