Test fails on s390x

[misery]

Describe the bug
Tried to build a package for Alpine. It works without problems on my machine but it fails on s390x CI.
https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/7981

To Reproduce
Steps to reproduce the behavior:

1. Build croc for s390x
2. Run unit tests

Expected behavior
No failing tests. ;-)

Version
8.0.11

Additional context
https://gitlab.alpinelinux.org/misery/aports/-/jobs/118553

>>> croc: Unpacking /var/cache/distfiles/croc_8.0.11_src.tar.gz...
?   	github.com/schollz/croc/v8	[no test files]
?   	github.com/schollz/croc/v8/src/cli	[no test files]
ok  	github.com/schollz/croc/v8/src/comm	0.319s
ok  	github.com/schollz/croc/v8/src/compress	0.016s
ok  	github.com/schollz/croc/v8/src/croc	2.752s
ok  	github.com/schollz/croc/v8/src/crypt	0.017s
?   	github.com/schollz/croc/v8/src/install	[no test files]
ok  	github.com/schollz/croc/v8/src/message	0.429s
?   	github.com/schollz/croc/v8/src/models	[no test files]
ok  	github.com/schollz/croc/v8/src/tcp	0.471s
[172.17.0.4] <nil>
--- FAIL: TestIMOHashFile (0.14s)
    utils_test.go:68: 
        	Error Trace:	utils_test.go:68
        	Error:      	Not equal: 
        	            	expected: "c0d1e123a96a598ea801cc503d3db8c0"
        	            	actual  : "c0d1e123a419ea942bc9b06c97cfb64e"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-c0d1e123a96a598ea801cc503d3db8c0
        	            	+c0d1e123a419ea942bc9b06c97cfb64e
        	Test:       	TestIMOHashFile
--- FAIL: TestHashFile (0.00s)
    utils_test.go:164: 
        	Error Trace:	utils_test.go:164
        	Error:      	Not equal: 
        	            	expected: "18c9673a4bb8325d323e7f24fda9ae1e"
        	            	actual  : "1860fd5aeb1ac95051d1cc57840037a1"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-18c9673a4bb8325d323e7f24fda9ae1e
        	            	+1860fd5aeb1ac95051d1cc57840037a1
        	Test:       	TestHashFile
148.100.88.62
172.17.0.4
FAIL
FAIL	github.com/schollz/croc/v8/src/utils	1.025s
FAIL
>>> ERROR: croc: check failed

[schollz]

@misery Okay, my bet is that it has something to the generation of a random file that was used for the tests. I generated a random file with seed 0 on my computer, which are the hashes I precomputed. Perhaps generating a random file from seed 0 is not the same for s390x? In any case, I changed the tests to work on a reproducible file filled with the same character. Can you try it now?

[misery]

Thanks for fast reply. :-)

But it's still broken: https://gitlab.alpinelinux.org/misery/aports/-/jobs/118672

By the way... armv7 crashes now.
https://gitlab.alpinelinux.org/misery/aports/-/jobs/118675

Perhaps generating a random file from seed 0 is not the same for s390x?

Well, I don't know s390x. :-) I just saw the failure in Alpines CI.

[misery]

Looks like mips64 is also broken.
https://build.alpinelinux.org/buildlogs//build-edge-mips64/testing/croc/croc-8.0.11-r0.log

Is it a flaky test or is the test environment broken?

[theStack]

My guess here would be that those failing tests are caused by endianness problems, as the common denominator for the affected architectures (s390x, mips64) is that they use big endian byte ordering compared to the much more common little endian. Maybe imohash is not endian-agnostic?

[misery]

It is still failing with 8.6.12 on s390x and mips/mips64

?   	github.com/schollz/croc/v8	[no test files]
?   	github.com/schollz/croc/v8/src/cli	[no test files]
ok  	github.com/schollz/croc/v8/src/comm	0.387s
ok  	github.com/schollz/croc/v8/src/compress	0.147s
ok  	github.com/schollz/croc/v8/src/croc	2.942s
ok  	github.com/schollz/croc/v8/src/crypt	0.022s
?   	github.com/schollz/croc/v8/src/install	[no test files]
ok  	github.com/schollz/croc/v8/src/message	0.493s
?   	github.com/schollz/croc/v8/src/models	[no test files]
ok  	github.com/schollz/croc/v8/src/tcp	0.761s
[172.17.0.2] <nil>
--- FAIL: TestHashFile (0.00s)
    utils_test.go:169: 
        	Error Trace:	utils_test.go:169
        	Error:      	Not equal: 
        	            	expected: "18c9673a4bb8325d323e7f24fda9ae1e"
        	            	actual  : "1860fd5aeb1ac95051d1cc57840037a1"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-18c9673a4bb8325d323e7f24fda9ae1e
        	            	+1860fd5aeb1ac95051d1cc57840037a1
        	Test:       	TestHashFile
148.100.88.62
172.17.0.2
FAIL
FAIL	github.com/schollz/croc/v8/src/utils	0.480s
FAIL

https://gitlab.alpinelinux.org/misery/aports/-/jobs/371363

[kalafut]

Maybe imohash is not endian-agnostic?

@theStack @misery Given this issue around the murmur3 hash used within imohash, I think that's a reasonable assumption. I'd been considering swapping the murmur3 implementation to something like this one from twmb which claims stable behavior across endianess. This is a good nudge.

[kalafut]

I just pushed github.com/kalafut/[email protected] which uses the alternate hashing library and should be endian-agnostic. The croc tests all pass fine without change. I don't have a good way to test if this does really fix the endianness issue though (but it should).

[misery]

@kalafut Thanks, it seems to work.... https://gitlab.alpinelinux.org/misery/aports/-/jobs/377230