Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky tests #187

Open
2 of 4 tasks
shazow opened this issue Aug 6, 2016 · 4 comments
Open
2 of 4 tasks

Fix flaky tests #187

shazow opened this issue Aug 6, 2016 · 4 comments

Comments

@shazow
Copy link
Owner

shazow commented Aug 6, 2016

  • --- FAIL: TestHostNameCollision (0.01s)
    host_test.go:74: First client failed to get 'foo' name: " \* foo joined. (Connected: 1)" c57a5ac
  • --- FAIL: TestIgnore (0.00s)
    room_test.go:148: should have messages
  • --- FAIL: TestSetExpiring (0.00s)
    set_test.go:78: b-prefix incorrect: ["barbar" "2017-05-11 11:54:27.161591982 -0400 EDT"]
  • --- FAIL: TestHostGetPrompt (0.00s)
    host_test.go:44: Got: "[\x1b[38;05;172mfoo\x1b[0m] "; Expected: "[\x1b[38;05;88mfoo\x1b[0m] "
sahilgoel1610 added a commit to sahilgoel1610/ssh-chat that referenced this issue Aug 27, 2017
shazow pushed a commit that referenced this issue Aug 28, 2017
@tacerus
Copy link

tacerus commented Mar 9, 2021

Hello!

Is there an update on this?

I am working on a RPM package for ssh-chat, and encounter FAIL: TestHostGetPrompt on the architectures armv7l and i586 (which are not super important, as most people are likely using it on x86_64, but it would still be nice to widen the target audience - and I don't feel like disabling tests for those two architectures, if they work with x86_64 just fine). If you are interested in checking it out, you can find the build logs in https://build.opensuse.org/package/show/home:crameleon/ssh-chat (so far only SLES and openSUSE are relevant, still working on adapting the spec to Fedora and CentOS, so those are expected to fail).

Cheers!

@shazow
Copy link
Owner Author

shazow commented Mar 9, 2021

@tacerus Hiya, thanks for following up! I appreciate you maintaining this distro package downstream. :)

We did make progress on x86 as you noticed, go test -test.timeout 5s -count 100 ./... works reliably.

I'm not certain why arm7l and i586 would still have issues. One suggestion: Could try running it with/without -race, I recall that have a side effect on whether some tests were flakey before or not.

For what it's worth, it's not a super important test, the flake is simply messages coming in out of order. If you'd like to disable it for those builds, I think it's reasonable.

@tacerus
Copy link

tacerus commented Mar 9, 2021

Thanks for the input and headsup!

[ 47s] go test: -race is only supported on linux/amd64, linux/ppc64le, linux/arm64, freebsd/amd64, netbsd/amd64, darwin/amd64, darwin/arm64, and windows/amd64

I found that the tests pass if I run them only against /cmd/ssh-chat and will hence go ahead and implement a clause that will do that only for the problematic architectures.

@tacerus
Copy link

tacerus commented Mar 9, 2021

I got excited too early, after some testing this workaround showed only reliable on i586, on armv7l it would still fail. I hence implemented the cmd/ssh-chat workaround for i586 and skipped tests completely on armv7l, adding a note referring this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants