Reduce size of test-runner args

[andre15silva]

If we specify all test methods that are to be ran in a big enough project, we will easily reach the space available for command arguments set by Linux.

This limit can be adjusted, but even with 65MB available there is still an issue for thousands of methods.

Trace:

java.lang.RuntimeException: java.io.IOException: Cannot run program "java" (in directory "/home/andre/Repos/flacoco/."): error=7, Argument list too long
	at eu.stamp_project.testrunner.EntryPoint.runGivenCommandLine(EntryPoint.java:602)
	at eu.stamp_project.testrunner.EntryPoint.runCoveredTestResultPerTestMethods(EntryPoint.java:496)
	at fr.spoonlabs.flacoco.core.coverage.framework.JUnit4Strategy.execute(JUnit4Strategy.java:23)
	at fr.spoonlabs.flacoco.core.coverage.CoverageRunner.getCoverageMatrix(CoverageRunner.java:34)
	at fr.spoonlabs.flacoco.localization.spectrum.SpectrumRunner.computeCoverageMatrix(SpectrumRunner.java:37)
	at fr.spoonlabs.flacoco.localization.spectrum.SpectrumRunner.run(SpectrumRunner.java:22)
	at fr.spoonlabs.flacoco.api.Flacoco.runDefault(Flacoco.java:32)
	at fr.spoonlabs.flacoco.cli.FlacocoMain.call(FlacocoMain.java:105)
	at fr.spoonlabs.flacoco.cli.FlacocoMain.call(FlacocoMain.java:25)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at fr.spoonlabs.flacoco.cli.FlacocoMain.main(FlacocoMain.java:96)
Caused by: java.io.IOException: Cannot run program "java" (in directory "/home/andre/Repos/flacoco/."): error=7, Argument list too long
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at eu.stamp_project.testrunner.EntryPoint.runGivenCommandLine(EntryPoint.java:591)
	... 16 more
Caused by: java.io.IOException: error=7, Argument list too long
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 17 more

[andre15silva]

This has been achieved, for now, by passing only the test class names.

[martinezmatias]

This has been achieved, for now, by passing only the test class names

It makes sense to pass the class name. I would not think that this would impact on the output.

Another option is to change the test-runnerin order to add the possibility to pass as argument a file containing all the method/class names

[andre15silva]

It makes sense to pass the class name. I would not think that this would impact on the output.

Yes, and theoretically it should work as our detection mechanism is based on the one from test-runner.

It is still useful for us so that we can identify which strategies to run which classes.

Another option is to change the test-runnerin order to add the possibility to pass as argument a file containing all the method/class names

That could be a solution yes. Opened STAMP-project/test-runner#107

[martinezmatias]

Yes, and theoretically it should work as our detection mechanism is based on the one from test-runner.

Exact!

That could be a solution yes. Opened STAMP-project/test-runner#107

Perfect. Thanks!

[andre15silva]

Another problem related to this issue.

In the #86 CI, Windows builds that run testMath70 fail due to the size of the args (in this case, probably the new addition, which is the includes option for jacoco).

https://github.com/SpoonLabs/flacoco/runs/3408893104?check_suite_focus=true