Handle empty memory type in agent configuration

[arjunkumargiri]

Description

Handle empty memory type in flow agent configuration. Note that conversational agent still requires memory type to be defined.

Test agent configuration

POST _plugins/_ml/agents/_register
{
  "name": "Test_Agent_For_RAG",
  "type": "flow",
  "description": "this is a test agent",
  "app_type": "my app",
  "tools": [
      {
      "type": "CatIndexTool",
      "name": "CatIndexTool",
      "description": "Use this tool to get OpenSearch index information: (health, status, index, uuid, primary count, replica count, docs.count, docs.deleted, store.size, primary.store.size).",
      "include_output_in_agent_response": true
    },
    {
      "type": "VectorDBTool",
      "name": "PopulationDataTool",
      "description": "Use this tool to fetch population information",
      "parameters": {
        "model_id": "6nrN2YsBn9PZfduo9pOo",
        "index": "my_test_data_1",
        "embedding_field": "embedding",
        "source_field": ["text"]
      }
    },
    {
      "type": "MLModelTool",
      "description": "A general tool to answer any question",
      "parameters": {
        "model_id": "QahqWowBeu5xSCcODPNw",
        "prompt": "\n\nHuman:You are a professional data analysist. You will always answer question based on the given context first. If the answer is not directly shown in the context, you will analyze the data and find the answer. If you don't know the answer, just say don't know. \n\n Context:\n${parameters.PopulationDataTool.output}\n\nHuman:${parameters.question}\n\nAssistant:"
      }
    }
  ]
}

Test request

POST /_plugins/_ml/agents/R6huWowBeu5xSCcOoPM4/_execute
{
    "parameters": {
        "question": "What is the population in Seattle?",
        "verbose": false
    }
}

Response

{
  "inference_results": [
    {
      "output": [
        {
          "name": "CatIndexTool",
          "result": """health	status	index	uuid	pri	rep	docs.count	docs.deleted	store.size	pri.store.size
yellow	open	.plugins-ml-memory-message	S_bi7-qaSmeyAC670rQPKw	1	1	978	8	377.2kb	377.2kb
green	open	.plugins-ml-model-group	Ord6f9WNRk6y4Otgj0Omxg	1	0	2	0	10.9kb	10.9kb
yellow	open	.plugins-ml-memory-meta	e_q4R99IQQK3pC1sWZwkJA	1	1	139	6	38kb	38kb
yellow	open	.plugins-ml-conversation-meta	kc-ANGRXQDSU9mqq3g-MQw	1	1	56	0	19.2kb	19.2kb
green	open	.ql-datasources	hqwDvsYLTyOVII30K604ZQ	1	0	0	0	208b	208b
green	open	.plugins-ml-agent	LC2Z2auCRmeCm9aR-IyHPA	1	0	99	0	325.1kb	325.1kb
green	open	.plugins-ml-task	aSFfJGToRJi5ddiSeSS8sA	1	0	87	5	107.7kb	107.7kb
green	open	.plugins-ml-connector	9EeGaG8cTt-owlefp-0zvQ	1	0	13	0	247.2kb	247.2kb
green	open	.kibana_2	nj7tlCwrQnOCO0CAj63vrA	1	0	7	0	16.6kb	16.6kb
green	open	.kibana_1	tC1y7PXRSK6NYVrTzzPXJw	1	0	6	0	26.6kb	26.6kb
green	open	.kibana_3	xN3ybMj0SXO_FCmfgBku8Q	1	0	7	0	11.7kb	11.7kb
green	open	.plugins-ml-config	zENfAIpFQHyxATLkpAV1Ag	1	0	1	0	3.9kb	3.9kb
green	open	.opensearch-observability	uqEoBMIaQ5eSLOqIknsBUg	1	0	0	0	208b	208b
green	open	.plugins-ml-model	69N8PBd_RZa73tF4umwUlw	1	0	27	9	170.4mb	170.4mb
yellow	open	my_test_data	AnsDyfxjSFeO-t-Zi8YmQg	1	1	0	0	208b	208b
yellow	open	.plugins-ml-conversation-interactions	fX-ZMD65S0qeIvzshm8HXw	1	1	14	1	41.5kb	41.5kb
yellow	open	my_test_data_1	qDV348DBSMKGkWX1sqGtWQ	1	1	6	0	45.7kb	45.7kb
"""
        },
        {
          "result": " Based on the given context, there is no information provided about the population in Seattle. The context only contains information about the populations of Chicago and Miami metro areas. Since the answer is not directly shown in the context, I would say I don't know the population in Seattle."
        }
      ]
    }
  ]
}

Issues Resolved

[List any issues this PR will resolve]

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: 8 lines in your changes are missing coverage. Please review.

Comparison is base (44118ef) 68.95% compared to head (f998c9f) 68.92%.

Files	Patch %	Lines
...ch/ml/engine/algorithms/agent/MLAgentExecutor.java	0.00%	5 Missing ⚠️
.../ml/engine/algorithms/agent/MLFlowAgentRunner.java	40.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@                        Coverage Diff                        @@
##             feature/agent_framework_dev    #1745      +/-   ##
=================================================================
- Coverage                          68.95%   68.92%   -0.03%     
  Complexity                          2611     2611              
=================================================================
  Files                                241      241              
  Lines                              12880    12882       +2     
  Branches                            1291     1292       +1     
=================================================================
- Hits                                8881     8879       -2     
- Misses                              3393     3395       +2     
- Partials                             606      608       +2     

Flag	Coverage Δ
ml-commons	68.92% <20.00%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ylwu-amzn]

Can you also test multiple tools use case?

[arjunkumargiri]

Can you also test multiple tools use case?

Updated PR description with testing details using multiple tools.

[ylwu-amzn]

Thanks.

Question not related to this bug fix: for multi-tool use case, we will return response of each tool? I think we should keep it same with chat agent, which outputs the final answer as response, but user can add the tool response to additional info.

[arjunkumargiri]

Agreed. We will need to finalize on agent execution response schema. Will move Tool output of flow agent to ModelTensor#dataAsMap as part of a different PR.