[Question]: Should a FunctionCallingAgent built using a BedrockConverse LLM respect allow_parallel_tool_calls flag?

[kinano]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

I am using BedrockConverse to init an LLM and FunctionCallingAgent to init an agent with multiple tools as follows:

from llama_index.core.agent import FunctionCallingAgent
from llama_index.llms.bedrock_converse import BedrockConverse
# other imports...

llm = BedrockConverse(
        model=model,
        profile_name=profile_name,
        max_tokens=4096,
        temperature=0, 
        additional_kwargs={
            "performanceConfig": {
                "latency": "optimized",  # https://docs.aws.amazon.com/bedrock/latest/userguide/latency-optimized-inference.html
            },
        },
        system_prompt=system_prompt,
    )

tools = [tool_1, tool_2, tool_3, tool_4, tool_5]
agent = FunctionCallingAgent.from_tools(
   tools=tools,
   llm=get_llm(system_prompt=SYSTEM_PROMPT),
   verbose=verbose,
   allow_parallel_tool_calls=True,
)

res = agent.chat(
   message=prompt,
   chat_history=chat_history,
)

I noticed that if the prompt requires multiple tool calls, the tools are executed one by one sequentially despite passing the paramter allow_parallel_tool_calls=True to FunctionCallingAgent.from_tools.

I traced how the flag allow_parallel_tool_calls is passed throughout the call chain and noticed that it is seemingly ignored by BedrockConverse._prepare_chat_with_tools
Source code of that method pasted here for ease of following:

    def _prepare_chat_with_tools(
        self,
        tools: List["BaseTool"],
        user_msg: Optional[Union[str, ChatMessage]] = None,
        chat_history: Optional[List[ChatMessage]] = None,
        verbose: bool = False,
        allow_parallel_tool_calls: bool = False,
        tool_choice: Optional[dict] = None,
        **kwargs: Any,
    ) -> Dict[str, Any]:
        """Prepare the arguments needed to let the LLM chat with tools."""
        chat_history = chat_history or []

        if isinstance(user_msg, str):
            user_msg = ChatMessage(role=MessageRole.USER, content=user_msg)
            chat_history.append(user_msg)

        # convert Llama Index tools to AWS Bedrock Converse tools
        tool_config = tools_to_converse_tools(tools)
        if tool_choice:
            # https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ToolChoice.html
            # e.g. { "auto": {} }
            tool_config["toolChoice"] = tool_choice

        return {
            "messages": chat_history,
            "tools": tool_config,
            **kwargs,
        }

My questions are:

1. Am I passing allow_parallel_tool_calls correctly above or am I missing something?
2. Should BedrockConverse respect this flag? Do we have a bug here?

[dosubot]

1. You are passing the allow_parallel_tool_calls flag correctly in the initialization of the FunctionCallingAgent:

agent = FunctionCallingAgent.from_tools(
   tools=tools,
   llm=get_llm(system_prompt=SYSTEM_PROMPT),
   verbose=verbose,
   allow_parallel_tool_calls=True,
)

This indicates that you intend for the tools to be called in parallel.

2. Yes, the BedrockConverse LLM should respect the allow_parallel_tool_calls flag when using FunctionCallingAgent with multiple tools. However, it seems that the _prepare_chat_with_tools method in BedrockConverse does not currently utilize this flag, which might be why the tools are executed sequentially. This could indicate a bug or an oversight in the implementation, as the flag should influence whether tools are called in parallel or sequentially ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[kinano]

I found a solution to this by leveraging help from llama-index discord group (AI powered) and an AWS Bedrock expert who joined us on a call today. Apparently, Claude may not support parallel tool calling 🫠. On the other hand, AWS Nova Pro seems to support parallel tool calling.