DRAFT: Add script to build tests

[Mab879]

Description:

This is a very rough draft a script that renders the Automatus tests for a given product. The script will be clean up once the general form is agreed upon.

Rationale:

Make testing with thin data streams possible.

Review Hints:

1. ./build_product rhel10
2. python ./build-scripts/build_tests.py --build-config-yaml build/build_config.yml --product-yaml products/rhel10/product.yml --output build/rhel10/tests --resolved-rules-dir build/rhel10/rules

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[Mab879]

Things left to do:

1. Check on Automatus runs. They seem to be broken.
   Commands like ./automatus.py rule --datastream ../build/ssg-rhel10-ds.xml --libvirt qemu:///system automatus_rhel10 root_permissions_syslibrary_files are failing.

2. I need to doc how to turn it on.

3. Possible turn down the jobs number when building many products

4. Verify the correct tests are being ran

[comps]

str.endswith() takes a tuple, so you can probably just

Suggested change

	return (filename.endswith('.pass.sh')
	or filename.endswith('.fail.sh') or
	filename.endswith('.notapplicable.sh'))
	return filename.endswith(('.pass.sh', '.fail.sh', '.notapplicable.sh'))

[codeclimate]

Code Climate has analyzed commit 99e5f9b and detected 5 issues on this pull request.

Here's the issue category breakdown:

Category	Count
Complexity	4
Style	1

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 61.9% (0.0% change).

View more on Code Climate.

[comps]

If you don't mind interleaving the split, you could use python's array slicing "step" property.

Given 3 workers and a total of 10 rules, you can assign rules to them like this:

>>> a = [1,2,3,4,5,6,7,8,9,10]
>>> a[0::3]
[1, 4, 7, 10]
>>> a[1::3]
[2, 5, 8]
>>> a[2::3]
[3, 6, 9]

where the first (start) number is worker index, and the second (step) number is the total number of workers.

This is what we use in Contest to avoid any off-by-one problems and ensure no rule is left behind, while making things just simpler and evenly distributing rule complexity across all workers (as complex/simple rules tend to alphabetically bunch up).

https://github.com/RHSecurityCompliance/contest/blob/1204f812956a855eb15d26b1014cb28e5306ee05/per-rule/test.py#L93-L97