Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky Test: toStringWithUnprocessedProps due to Strict JSON Ordering #515

Open
zhihao11ui opened this issue Oct 8, 2024 · 2 comments
Open

Comments

@zhihao11ui
Copy link
Contributor

I encountered a potential flaky test when running the following command: mvn edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=org.everit.json.schema.ObjectSchemaTest#toStringWithUnprocessedProps -DfailIfNoTests=false

The test toStringWithUnprocessedProps fails intermittently because it uses the sameJsonAs function, which ultimately relies on deepEqualArrays to compare JSON objects. This function enforces strict order of elements in the JSON arrays. As a result, this test is prone to failure when the order of elements is altered, even if the content is otherwise valid.

Suggested Fix:
I propose switching the JSON comparison in this test to use JSONAssert, which is more tolerant of ordering differences in JSON arrays while still ensuring the content is correct. This should resolve the flaky behavior and make the test more reliable across different runs.

I will submit a pull request to apply this change. Let me know if any further details are needed.

@erosb
Copy link
Contributor

erosb commented Oct 8, 2024

Hello, what does 'potential" mean? Did it actually ever fail for you?

@zhihao11ui
Copy link
Contributor Author

Hello, it will always fail because this is a flaky test, sorry for the confusion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants