Add dependent plugins validation during plugin load

[junqiu-lei]

Description

Add dependent plugins validation during neural search plugin load stage.

Related Issues

Closes #480

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[heemin32]

Thanks for the PR.

Could you confirm it will fail in both cases?

1. Attempting to install the neural plugin without having the ml-common plugin already installed.
2. Installing both the ml-common and neural plugins, then removing the ml-common plugin and restarting the cluster.

Because we added knn and ml common code as dependency through gradle, this Class.forName(org.opensearch.knn.plugin.KNNPlugin) might fail to check the actual installation of the plugin in the OpenSearch cluster.

[yuye-aws]

I do not have much context. Can you please explain why org.opensearch.knn.plugin.KNNPlugin and org.opensearch.ml.client.MachineLearningClient still needs check even if both opensearch-knn and opensearch-ml-commons are available

[heemin32]

opensearch-knn is just a plugin name we are using in a log and org.opensearch.knn.plugin.KNNPlugin is the actual class that we are validating its existence.

I think this code might look confusing. Can we change to something like this?

public enum DependentPlugin {
  private String klass;

  ML_COMMON("org.opensearch.ml.client.MachineLearningClient"),
  KNN("org.opensearch.knn.plugin.KNNPlugin");

  DependentPlugin(final String klass) {
    this.klass = klass;
  }

  public void validateAvailability() {
    try {
               Class.forName(klass);
               log.info("Plugin [{}] is installed", this.name());
     } catch (ClassNotFoundException e) {
                throw new IllegalStateException("Neural Search plugin requires the " + this.name() + " plugin to be installed", e);
     }
  }
}

[junqiu-lei]

Updated to use enum DependentPlugin.

Few suggestions were added

[junqiu-lei]

Thanks for the PR.

Could you confirm it will fail in both cases?

1. Attempting to install the neural plugin without having the ml-common plugin already installed.
2. Installing both the ml-common and neural plugins, then removing the ml-common plugin and restarting the cluster.

Because we added knn and ml common code as dependency through gradle, this Class.forName(org.opensearch.knn.plugin.KNNPlugin) might fail to check the actual installation of the plugin in the OpenSearch cluster.

@heemin32 I found that at cluster install plugin stage, the validation from this PR can be passed even the ml-commons plugin is removed from OpenSearch bundle.

Going to close this PR, will raise another PR by adding ml-commons in the neural search extended plugin, which will require code changes in k-NN and ml-commons plugins to use some libs by compile only, this way can resolve all OpenSearch jar hell checks.