Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Fix OSB's Integration tests #738

Open
OVI3D0 opened this issue Jan 29, 2025 · 0 comments
Open

[Bug]: Fix OSB's Integration tests #738

OVI3D0 opened this issue Jan 29, 2025 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@OVI3D0
Copy link
Member

OVI3D0 commented Jan 29, 2025

Describe the bug

Recently, OpenSearch was updated to Lucene 10, which included updating the minimum JDK version to 21 for gradle operations instead of JDK 11. This began causing failures in OSB's integration tests since we use JDK 17:

osbenchmark.exceptions.BuildError: Executing './gradlew clean' failed. The last 20 lines in the build log file are:
=========================================================================================================
	* What went wrong:
	A problem occurred evaluating project ':buildSrc'.
	> At least Java 21 is required to build opensearch gradle tools

This solution was introduced temporarily to swap between JDK versions when needed and run the integration tests, but this is only a fix for integration tests run using Github Actions, and does not help if users want to run their tests locally using make it.

Another solution proposed was to add JDK 21 to the provision_config resources in OSB, but this caused different failures referenced here:

https://github.com/opensearch-project/opensearch-benchmark/actions/runs/13001769222/job/36261656956#step:11:243
https://github.com/opensearch-project/opensearch-benchmark/actions/runs/12999003364/job/36253370447#step:9:145

More investigation is needed to figure out what the best solution is going forward.

To reproduce

Run integration tests using GH actions

Expected behavior

Integration tests run normally without JDK failures.

Relevant log output

When attempting to use JDK 21 for gradle operations and JDK 17 for opensearch operations:

Error:  Cannot execute-test. JAVA_HOME points to JDK 17 but it should point to JDK 21.
	Traceback (most recent call last):
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/builder/builder.py", line 576, in receiveMsg_StartNodes
    self.builder.start_engine()
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/builder/builder.py", line 691, in start_engine
    binaries = self.supply()
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/builder/supplier.py", line 293, in __call__
    supplier.prepare()
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/builder/supplier.py", line 366, in prepare
    self.source_supplier.prepare()
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/builder/supplier.py", line 403, in prepare
    self.builder.build([
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/builder/supplier.py", line 696, in build
    self.run(command, override_src_dir)
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/builder/supplier.py", line 705, in run
    build_cmd = "export JAVA_HOME={}; cd {}; {} > {} 2>&1".format(self.java_home, src_dir, command, log_file)
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/builder/supplier.py", line 691, in java_home
    _, self._java_home = jvm.resolve_path(self.build_jdk)
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/utils/jvm.py", line 123, in resolve_path
    return majors, _resolve_single_path(majors, sysprop_reader=sysprop_reader)
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/utils/jvm.py", line 163, in _resolve_single_path
    java_home = do_resolve(generic_env_var, major)
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py38/lib/python3.8/site-packages/osbenchmark/utils/jvm.py", line 150, in do_resolve
    raise exceptions.SystemSetupError("{} points to JDK {} but it should point to JDK {}.".format(env_var, actual_major, major))
osbenchmark.exceptions.SystemSetupError: JAVA_HOME points to JDK 17 but it should point to JDK 21.

When attempting to only use JDK 21 for both gradle operations and opensearch operations:

2025-01-27T21:57:36 Invoking OSB: opensearch-benchmark execute-test --distribution-version="1.3.9" --workload="geonames" --test-mode --provision-config-instance=4gheap --target-hosts=127.0.0.1:19200 --kill-running-processes --on-error='abort' --configuration-name='in-memory-it'
Error:  Cannot execute-test. pid file not available after 60 seconds!
	Traceback (most recent call last):
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py310/lib/python3.10/site-packages/osbenchmark/builder/builder.py", line 576, in receiveMsg_StartNodes
    self.builder.start_engine()
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py310/lib/python3.10/site-packages/osbenchmark/builder/builder.py", line 695, in start_engine
    self.nodes = self.launcher.start(self.node_configs)
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py310/lib/python3.10/site-packages/osbenchmark/builder/launcher.py", line 140, in start
    return [self._start_node(node_configuration, node_count_on_host) for node_configuration in node_configurations]
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py310/lib/python3.10/site-packages/osbenchmark/builder/launcher.py", line 140, in <listcomp>
    return [self._start_node(node_configuration, node_count_on_host) for node_configuration in node_configurations]
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py310/lib/python3.10/site-packages/osbenchmark/builder/launcher.py", line 172, in _start_node
    node_pid = self._start_process(binary_path, env)
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py310/lib/python3.10/site-packages/osbenchmark/builder/launcher.py", line 236, in _start_process
    return wait_for_pidfile(io.escape_path(os.path.join(".", "pid")))
  File "/home/runner/work/opensearch-benchmark/opensearch-benchmark/.tox/py310/lib/python3.10/site-packages/osbenchmark/builder/launcher.py", line 123, in wait_for_pidfile
    raise exceptions.LaunchError(msg)
osbenchmark.exceptions.LaunchError: pid file not available after 60 seconds!
@OVI3D0 OVI3D0 added bug Something isn't working untriaged labels Jan 29, 2025
@OVI3D0 OVI3D0 self-assigned this Jan 30, 2025
@OVI3D0 OVI3D0 changed the title [Bug]: Fix Integration tests [Bug]: Fix OSB's Integration tests Jan 30, 2025
@gkamat gkamat assigned gkamat and unassigned OVI3D0 Jan 30, 2025
@gkamat gkamat removed the untriaged label Jan 30, 2025
@gkamat gkamat moved this from 🆕 New to 🏗 In progress in Engineering Effectiveness Board Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: 🏗 In progress
Development

No branches or pull requests

2 participants