Update documentation of ml inference processors to support for local models

[rbhavna]

Description

Currently ml inference processor is only supporting remote models. Starting from 2.15 release, it will start to support local models.

Related PR: opensearch-project/ml-commons#2508

Issues Resolved

Closes #7354

Version

2.15

Checklist

• By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
  For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[mingshl]

For more information on local models,

[rbhavna]

good catch, thanks

[ohltyler]

Not particularly about the local model feature, but one thing related to the ml inference processors I've found confusing is the input_map parameter. Currently, for remote models, in predict, you need to explicitly pass input fields within a base parameters field. However, in the input_map, you only specify the input fields, and ignore parameters altogether. Example:

Given some connector with this request body:

      "request_body": """{ "texts": ${parameters.texts}, "truncate": "END" }"""

Predict will succeed, passing the base parameters field:

POST /_plugins/_ml/models/abc123/_predict
{
  "parameters": {
    "texts": [
      "hello",
      "world"
      ]
  }
}

But the inference processor must be configured using only the nested texts field, not parameters.texts which was my assumption based on the documentation:

{
        "ml_inference": {
          "model_id": "abc123",
          "input_map": [
            {
              "texts": "my-input"
            }
          ],
        }
      }

Could we add some clarity around the keys for the input map to be the fields within the base parameters field?

cc @mingshl

[ylwu-amzn]

Local model ?

[ylwu-amzn]

minor: move this config parameter to after line 31 ?

[ylwu-amzn]

@rbhavna Have you verified this is optional for both remote and local model?

[rbhavna]

Sorry this is required for local models. Will update the documentation

[ylwu-amzn]

Suggest also add "target_response": ["sentence_embedding"]

POST /_plugins/_ml/_predict/text_embedding/WWQI44MBbzI2oUKAvNUt
{
  "text_docs":[ "today is sunny", "hello world"],
  "return_number": true,
  "target_response": ["sentence_embedding"]
}

[ylwu-amzn]

Seems this is not for local model, which doesn't have dataAsMap

[ylwu-amzn]

Better to explain which model you are using , suggest use one of pretrained models

[rbhavna]

Not particularly about the local model feature, but one thing related to the ml inference processors I've found confusing is the input_map parameter. Currently, for remote models, in predict, you need to explicitly pass input fields within a base parameters field. However, in the input_map, you only specify the input fields, and ignore parameters altogether. Example:

Given some connector with this request body:

      "request_body": """{ "texts": ${parameters.texts}, "truncate": "END" }"""

Predict will succeed, passing the base parameters field:

POST /_plugins/_ml/models/abc123/_predict
{
  "parameters": {
    "texts": [
      "hello",
      "world"
      ]
  }
}

But the inference processor must be configured using only the nested texts field, not parameters.texts which was my assumption based on the documentation:

{
        "ml_inference": {
          "model_id": "abc123",
          "input_map": [
            {
              "texts": "my-input"
            }
          ],
        }
      }

Could we add some clarity around the keys for the input map to be the fields within the base parameters field?

cc @mingshl

Not particularly about the local model feature, but one thing related to the ml inference processors I've found confusing is the input_map parameter. Currently, for remote models, in predict, you need to explicitly pass input fields within a base parameters field. However, in the input_map, you only specify the input fields, and ignore parameters altogether. Example:

Given some connector with this request body:

      "request_body": """{ "texts": ${parameters.texts}, "truncate": "END" }"""

Predict will succeed, passing the base parameters field:

POST /_plugins/_ml/models/abc123/_predict
{
  "parameters": {
    "texts": [
      "hello",
      "world"
      ]
  }
}

But the inference processor must be configured using only the nested texts field, not parameters.texts which was my assumption based on the documentation:

{
        "ml_inference": {
          "model_id": "abc123",
          "input_map": [
            {
              "texts": "my-input"
            }
          ],
        }
      }

Could we add some clarity around the keys for the input map to be the fields within the base parameters field?

cc @mingshl

Yes that is correct. User does not have to specify parameters in input_map. Since for remote models, all fields are to be passed inside parameters, we only let user specify the particular field. Internally, while passing to the remote model for inference, we do wrap all the input field texts and other config fields(if any) in parameters map and send as model input here.

[rbhavna]

Could you help simplify this part? We just say that we map book.*.chunk.text.*.context to model's input text_docs field. More details could be confusing to user sometime.

[rbhavna]

ingested could be confusing because this is what user sends but finally ingested document is the output of the pipeline.

[rbhavna]

Can we simplify here as well? The output_map maps the model's embeddings in the path $.inference_results.*.output.*.data to the new ingest document field specified by the path book.*.chunk.text.*.context_embedding. Also it need not be in the same path as context. Depends on user's configuration of the processor

[rbhavna]

@kolchfa-aws Thanks Fanit for adding all the details and making the documentation more clear. Overall, looks good to me

[ohltyler]

Usually, or will always be? From my understanding, it is always.

[ohltyler]

Since the ml inference processor makes this assumption within the mapping / conversion logic, I think we should make this a bit clearer.

[ohltyler]

cc @mingshl

[rbhavna]

We are specifying here to the user that they need not give parameters path here. Should we add more info?

[kolchfa-aws]

LGTM. Thank you, @rbhavna!

[natebower]

@kolchfa-aws @rbhavna Please see my comments and changes and let me know if you have any questions. Thanks!