Comparison of AccessMod runtime env + specs

[fxi]

Performance testing on Madagascar dataset:

146'000'000 cells, 2960 facilities, 120min anisotropic travel

os_host	disk_type	docker	vbox	version	cpu_n	ram_gb	duration_minutes
google	ssd	yes	no	v5.7.10-alpha	4	16	3.86
google	ssd	yes	no	v5.7.10-alpha	30	120	4.01
mac	ssd	yes	no	v5.7.10-alpha	8	8	4.1
mac	ssd	yes	yes	v5.7.10-alpha	8	8	12.2
mac	ssd	yes	yes	v5.7.10-alpha	2	4	22.0
mac	ssd	no	yes	v5.6.56	2	8	26.8
mac	ssd	no	yes	v5.6.56	2	4	27.6
mac	ssd	no	yes	v5.6.56	4	8	28.2
mac	ssd	no	yes	v5.6.56	2	2	52.52

[nicolasray]

Great, thanks! Just one important test missing:
mac ssd no yes v5.6.56 2 8 ...

We need to know if its the CPUs or the RAM (likely) that provides the best perf with the no-Docker Vbox version.

[nicolasray]

Wait... we should also do:
mac | ssd | no | yes | v5.6.56 | 2 | 4 | ...

because we also want a direct comparison with and without the Docker code, so that we can quantify the gain with that new code.

[fxi]

Updated with the requested config.

Notes:

• Performance dropped significantly when RAM < 3GB, which make sense given the estimated 2.7GB requested by r.walk. A lot of disk write ensued.
• Docker version featured a newer version of GRASS, which could explain the performance gain.
• Additional virtualisation of a dockerized image is a huge performance cost, but does better than current the current virtual machine version.

Caveats:

• Test done manually, results copy/pasted from the final report
• Unplanned testing session : tests specs are not uniform
• macOS benchmarks were done on a not isolated, running machine

To do :

• Automate testing, when the CLI is ready. Check with UNIGE or a cloud provider for an API compatible with docker-machine or similar.
• Confirm the performance gain on Windows with Docker for Windows ( e.g. using the latest launcher )
• Testing on bare-metal + linux host with >= 4GB Ram, >= 2CPUs : performance gain is expected to be huge