You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When calling a tool with a defined input schema, the LLM ignores this schema and makes up its own. If I include the schema in the prompt, then tool calling works as expected. It seems as thought the LLM isn't being shown the schema for the tool.
When you run it, you will see that you get errors similar to this trace:
Error Trace
{
"traceId": "3c524c9c0617d17a79f11c8741bc6a7c",
"displayName": "twentyQuestionsFlow",
"startTime": 1739895486270,
"endTime": 1739895487589.9666,
"spans": {
"4eed90ede4e9a7d7": {
"spanId": "4eed90ede4e9a7d7",
"traceId": "3c524c9c0617d17a79f11c8741bc6a7c",
"parentSpanId": "f51662ca8718e3c7",
"startTime": 1739895486290,
"endTime": 1739895487588.4597,
"attributes": {
"genkit:type": "util",
"genkit:name": "generate",
"genkit:path": "/{twentyQuestionsFlow,t:flow}/{generate,t:util}",
"genkit:input": "{\"model\":\"googleai/gemini-2.0-flash\",\"messages\":[{\"role\":\"system\",\"content\":[{\"text\":\"\\nYou are going to play the game of \\\"20 Questions\\\" with a user where they have something in mind and you are trying to guess what it is by asking a series of yes/no questions.\\n\\nYou'll receive a list of previous questions asked and their responses, and a list of your previous guesses.\\n\\nIf you have enough information to make a guess, just output a single string with your guess.\\n\\nIf you don't have enough information, formulate a question to ask the user to narrow down the possibilities using the askQuestionTool.\\n\\n\\nYou'll evaluate the result of asking the question with the askQuestionTool, and output a single string as a guess as to what the user is thinking of, or \\\"Still don't know\\\" if you still don't have enough information.\\n\\nPrevious questions:\\nQuestion: Is it alive?\\nResponse:\\n - no\\n\"}]},{\"role\":\"user\",\"content\":[{\"text\":\"\\nWhat am I thinking of?\"}]}],\"tools\":[\"/tool/askQuestionTool\"],\"config\":{\"responseMimeType\":\"application/json\"},\"output\":{\"format\":\"text\",\"jsonSchema\":null},\"maxTurns\":20}",
"genkit:state": "error"
},
"displayName": "generate",
"links": [],
"instrumentationLibrary": {
"name": "genkit-tracer",
"version": "v1"
},
"spanKind": "INTERNAL",
"sameProcessAsParentSpan": {
"value": true
},
"status": {
"code": 2,
"message": "INVALID_ARGUMENT: Schema validation failed. Parse Errors:\n\n- questionType: must be equal to one of the allowed values\n\nProvided data:\n\n{\n \"questionId\": \"1\",\n \"questionText\": \"Is it man-made?\",\n \"questionType\": \"boolean\"\n}\n\nRequired JSON schema:\n\n{\n \"type\": \"object\",\n \"properties\": {\n \"questionId\": {\n \"type\": \"string\",\n \"description\": \"Unique identifier for this question. Used to match responses. Must be unique across all questions within a session or interaction.\"\n },\n \"questionText\": {\n \"type\": \"string\",\n \"description\": \"The text of the question presented to the user. Should be clear and concise.\"\n },\n \"questionType\": {\n \"type\": \"string\",\n \"enum\": [\n \"multiple_choice\",\n \"short_answer\"\n ],\n \"description\": \"The type of question. Determines the expected format of the response. Yes/No questions are defined as multiple_choice questions with only two choices, with a minAllowed and maxAllowed of one.\"\n },\n \"choices\": {\n \"type\": \"object\",\n \"properties\": {\n \"minRequired\": {\n \"type\": \"number\",\n \"description\": \"The minimum number of answers required. Must be an integer between 1 and the number of choices, inclusive.\"\n },\n \"maxAllowed\": {\n \"type\": \"number\",\n \"description\": \"The maximum number of answers allowed. Must be an integer between 1 and the number of choices, inclusive. Must be greater than or equal to minRequired.\"\n },\n \"choices\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"value\": {\n \"type\": \"string\",\n \"description\": \"The value representing the choice (e.g., 'a', '1', 'option_one'). This value is used in the client's response to identify the selected choice(s). Should be concise and easily machine-readable.\"\n },\n \"label\": {\n \"type\": \"string\",\n \"description\": \"The human-readable label for the choice (e.g., 'Option One', 'Red', 'United States'). This is what's displayed to the user.\"\n },\n \"allowTextEntry\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"Whether or not there should be a text input associated with this choice. Typically used for an 'other' option in a multiple-choice question. Defaults to false.\"\n }\n },\n \"required\": [\n \"value\",\n \"label\"\n ],\n \"additionalProperties\": true\n },\n \"minItems\": 2,\n \"description\": \"Available choices for questions with a questionType of multiple_choice.\"\n }\n },\n \"required\": [\n \"minRequired\",\n \"maxAllowed\"\n ],\n \"additionalProperties\": true,\n \"description\": \"The choices for multiple_choice questions. Should not be defined for short_answer questions.\"\n }\n },\n \"required\": [\n \"questionId\",\n \"questionText\",\n \"questionType\"\n ],\n \"additionalProperties\": true,\n \"$schema\": \"http://json-schema.org/draft-07/schema#\"\n}"
},
"timeEvents": {
"timeEvent": [
{
"time": 1739895487588.3735,
"annotation": {
"attributes": {
"exception.type": "400",
"exception.message": "INVALID_ARGUMENT: Schema validation failed. Parse Errors:\n\n- questionType: must be equal to one of the allowed values\n\nProvided data:\n\n{\n \"questionId\": \"1\",\n \"questionText\": \"Is it man-made?\",\n \"questionType\": \"boolean\"\n}\n\nRequired JSON schema:\n\n{\n \"type\": \"object\",\n \"properties\": {\n \"questionId\": {\n \"type\": \"string\",\n \"description\": \"Unique identifier for this question. Used to match responses. Must be unique across all questions within a session or interaction.\"\n },\n \"questionText\": {\n \"type\": \"string\",\n \"description\": \"The text of the question presented to the user. Should be clear and concise.\"\n },\n \"questionType\": {\n \"type\": \"string\",\n \"enum\": [\n \"multiple_choice\",\n \"short_answer\"\n ],\n \"description\": \"The type of question. Determines the expected format of the response. Yes/No questions are defined as multiple_choice questions with only two choices, with a minAllowed and maxAllowed of one.\"\n },\n \"choices\": {\n \"type\": \"object\",\n \"properties\": {\n \"minRequired\": {\n \"type\": \"number\",\n \"description\": \"The minimum number of answers required. Must be an integer between 1 and the number of choices, inclusive.\"\n },\n \"maxAllowed\": {\n \"type\": \"number\",\n \"description\": \"The maximum number of answers allowed. Must be an integer between 1 and the number of choices, inclusive. Must be greater than or equal to minRequired.\"\n },\n \"choices\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"value\": {\n \"type\": \"string\",\n \"description\": \"The value representing the choice (e.g., 'a', '1', 'option_one'). This value is used in the client's response to identify the selected choice(s). Should be concise and easily machine-readable.\"\n },\n \"label\": {\n \"type\": \"string\",\n \"description\": \"The human-readable label for the choice (e.g., 'Option One', 'Red', 'United States'). This is what's displayed to the user.\"\n },\n \"allowTextEntry\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"Whether or not there should be a text input associated with this choice. Typically used for an 'other' option in a multiple-choice question. Defaults to false.\"\n }\n },\n \"required\": [\n \"value\",\n \"label\"\n ],\n \"additionalProperties\": true\n },\n \"minItems\": 2,\n \"description\": \"Available choices for questions with a questionType of multiple_choice.\"\n }\n },\n \"required\": [\n \"minRequired\",\n \"maxAllowed\"\n ],\n \"additionalProperties\": true,\n \"description\": \"The choices for multiple_choice questions. Should not be defined for short_answer questions.\"\n }\n },\n \"required\": [\n \"questionId\",\n \"questionText\",\n \"questionType\"\n ],\n \"additionalProperties\": true,\n \"$schema\": \"http://json-schema.org/draft-07/schema#\"\n}",
"exception.stacktrace": "GenkitError: INVALID_ARGUMENT: Schema validation failed. Parse Errors:\n\n- questionType: must be equal to one of the allowed values\n\nProvided data:\n\n{\n \"questionId\": \"1\",\n \"questionText\": \"Is it man-made?\",\n \"questionType\": \"boolean\"\n}\n\nRequired JSON schema:\n\n{\n \"type\": \"object\",\n \"properties\": {\n \"questionId\": {\n \"type\": \"string\",\n \"description\": \"Unique identifier for this question. Used to match responses. Must be unique across all questions within a session or interaction.\"\n },\n \"questionText\": {\n \"type\": \"string\",\n \"description\": \"The text of the question presented to the user. Should be clear and concise.\"\n },\n \"questionType\": {\n \"type\": \"string\",\n \"enum\": [\n \"multiple_choice\",\n \"short_answer\"\n ],\n \"description\": \"The type of question. Determines the expected format of the response. Yes/No questions are defined as multiple_choice questions with only two choices, with a minAllowed and maxAllowed of one.\"\n },\n \"choices\": {\n \"type\": \"object\",\n \"properties\": {\n \"minRequired\": {\n \"type\": \"number\",\n \"description\": \"The minimum number of answers required. Must be an integer between 1 and the number of choices, inclusive.\"\n },\n \"maxAllowed\": {\n \"type\": \"number\",\n \"description\": \"The maximum number of answers allowed. Must be an integer between 1 and the number of choices, inclusive. Must be greater than or equal to minRequired.\"\n },\n \"choices\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"value\": {\n \"type\": \"string\",\n \"description\": \"The value representing the choice (e.g., 'a', '1', 'option_one'). This value is used in the client's response to identify the selected choice(s). Should be concise and easily machine-readable.\"\n },\n \"label\": {\n \"type\": \"string\",\n \"description\": \"The human-readable label for the choice (e.g., 'Option One', 'Red', 'United States'). This is what's displayed to the user.\"\n },\n \"allowTextEntry\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"Whether or not there should be a text input associated with this choice. Typically used for an 'other' option in a multiple-choice question. Defaults to false.\"\n }\n },\n \"required\": [\n \"value\",\n \"label\"\n ],\n \"additionalProperties\": true\n },\n \"minItems\": 2,\n \"description\": \"Available choices for questions with a questionType of multiple_choice.\"\n }\n },\n \"required\": [\n \"minRequired\",\n \"maxAllowed\"\n ],\n \"additionalProperties\": true,\n \"description\": \"The choices for multiple_choice questions. Should not be defined for short_answer questions.\"\n }\n },\n \"required\": [\n \"questionId\",\n \"questionText\",\n \"questionType\"\n ],\n \"additionalProperties\": true,\n \"$schema\": \"http://json-schema.org/draft-07/schema#\"\n}\n at parseSchema (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/core/src/schema.ts:133:21)\n at AsyncFunction.actionFn.run (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/core/src/action.ts:282:13)\n at actionFn (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/core/src/action.ts:266:28)\n at resolveToolRequest (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/resolve-tool-requests.ts:118:26)\n at <anonymous> (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/resolve-tool-requests.ts:174:55)\n at Array.map (<anonymous>)\n at resolveToolRequests (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/resolve-tool-requests.ts:171:33)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at generate (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/action.ts:336:5)\n at <anonymous> (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/action.ts:130:22)"
},
"description": "exception"
}
}
]
}
},
"f51662ca8718e3c7": {
"spanId": "f51662ca8718e3c7",
"traceId": "3c524c9c0617d17a79f11c8741bc6a7c",
"startTime": 1739895486270,
"endTime": 1739895487589.9666,
"attributes": {
"genkit:type": "action",
"genkit:metadata:subtype": "flow",
"genkit:name": "twentyQuestionsFlow",
"genkit:isRoot": true,
"genkit:path": "/{twentyQuestionsFlow,t:flow}",
"genkit:metadata:context": "{}",
"genkit:input": "{\"includeSchema\":false,\"previousQuestions\":[{\"question\":{\"questionId\":\"q1\",\"questionText\":\"Is it alive?\",\"questionType\":\"multiple_choice\",\"choices\":{\"minRequired\":1,\"maxAllowed\":1,\"choices\":[{\"value\":\"yes\",\"label\":\"Yes\",\"allowTextEntry\":false},{\"value\":\"no\",\"label\":\"No\",\"allowTextEntry\":false}]}},\"response\":{\"questionId\":\"q1\",\"responses\":[\"no\"]}}]}",
"genkit:state": "error"
},
"displayName": "twentyQuestionsFlow",
"links": [],
"instrumentationLibrary": {
"name": "genkit-tracer",
"version": "v1"
},
"spanKind": "INTERNAL",
"sameProcessAsParentSpan": {
"value": true
},
"status": {
"code": 2,
"message": "INVALID_ARGUMENT: Schema validation failed. Parse Errors:\n\n- questionType: must be equal to one of the allowed values\n\nProvided data:\n\n{\n \"questionId\": \"1\",\n \"questionText\": \"Is it man-made?\",\n \"questionType\": \"boolean\"\n}\n\nRequired JSON schema:\n\n{\n \"type\": \"object\",\n \"properties\": {\n \"questionId\": {\n \"type\": \"string\",\n \"description\": \"Unique identifier for this question. Used to match responses. Must be unique across all questions within a session or interaction.\"\n },\n \"questionText\": {\n \"type\": \"string\",\n \"description\": \"The text of the question presented to the user. Should be clear and concise.\"\n },\n \"questionType\": {\n \"type\": \"string\",\n \"enum\": [\n \"multiple_choice\",\n \"short_answer\"\n ],\n \"description\": \"The type of question. Determines the expected format of the response. Yes/No questions are defined as multiple_choice questions with only two choices, with a minAllowed and maxAllowed of one.\"\n },\n \"choices\": {\n \"type\": \"object\",\n \"properties\": {\n \"minRequired\": {\n \"type\": \"number\",\n \"description\": \"The minimum number of answers required. Must be an integer between 1 and the number of choices, inclusive.\"\n },\n \"maxAllowed\": {\n \"type\": \"number\",\n \"description\": \"The maximum number of answers allowed. Must be an integer between 1 and the number of choices, inclusive. Must be greater than or equal to minRequired.\"\n },\n \"choices\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"value\": {\n \"type\": \"string\",\n \"description\": \"The value representing the choice (e.g., 'a', '1', 'option_one'). This value is used in the client's response to identify the selected choice(s). Should be concise and easily machine-readable.\"\n },\n \"label\": {\n \"type\": \"string\",\n \"description\": \"The human-readable label for the choice (e.g., 'Option One', 'Red', 'United States'). This is what's displayed to the user.\"\n },\n \"allowTextEntry\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"Whether or not there should be a text input associated with this choice. Typically used for an 'other' option in a multiple-choice question. Defaults to false.\"\n }\n },\n \"required\": [\n \"value\",\n \"label\"\n ],\n \"additionalProperties\": true\n },\n \"minItems\": 2,\n \"description\": \"Available choices for questions with a questionType of multiple_choice.\"\n }\n },\n \"required\": [\n \"minRequired\",\n \"maxAllowed\"\n ],\n \"additionalProperties\": true,\n \"description\": \"The choices for multiple_choice questions. Should not be defined for short_answer questions.\"\n }\n },\n \"required\": [\n \"questionId\",\n \"questionText\",\n \"questionType\"\n ],\n \"additionalProperties\": true,\n \"$schema\": \"http://json-schema.org/draft-07/schema#\"\n}"
},
"timeEvents": {
"timeEvent": [
{
"time": 1739895487589.763,
"annotation": {
"attributes": {
"exception.type": "400",
"exception.message": "INVALID_ARGUMENT: Schema validation failed. Parse Errors:\n\n- questionType: must be equal to one of the allowed values\n\nProvided data:\n\n{\n \"questionId\": \"1\",\n \"questionText\": \"Is it man-made?\",\n \"questionType\": \"boolean\"\n}\n\nRequired JSON schema:\n\n{\n \"type\": \"object\",\n \"properties\": {\n \"questionId\": {\n \"type\": \"string\",\n \"description\": \"Unique identifier for this question. Used to match responses. Must be unique across all questions within a session or interaction.\"\n },\n \"questionText\": {\n \"type\": \"string\",\n \"description\": \"The text of the question presented to the user. Should be clear and concise.\"\n },\n \"questionType\": {\n \"type\": \"string\",\n \"enum\": [\n \"multiple_choice\",\n \"short_answer\"\n ],\n \"description\": \"The type of question. Determines the expected format of the response. Yes/No questions are defined as multiple_choice questions with only two choices, with a minAllowed and maxAllowed of one.\"\n },\n \"choices\": {\n \"type\": \"object\",\n \"properties\": {\n \"minRequired\": {\n \"type\": \"number\",\n \"description\": \"The minimum number of answers required. Must be an integer between 1 and the number of choices, inclusive.\"\n },\n \"maxAllowed\": {\n \"type\": \"number\",\n \"description\": \"The maximum number of answers allowed. Must be an integer between 1 and the number of choices, inclusive. Must be greater than or equal to minRequired.\"\n },\n \"choices\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"value\": {\n \"type\": \"string\",\n \"description\": \"The value representing the choice (e.g., 'a', '1', 'option_one'). This value is used in the client's response to identify the selected choice(s). Should be concise and easily machine-readable.\"\n },\n \"label\": {\n \"type\": \"string\",\n \"description\": \"The human-readable label for the choice (e.g., 'Option One', 'Red', 'United States'). This is what's displayed to the user.\"\n },\n \"allowTextEntry\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"Whether or not there should be a text input associated with this choice. Typically used for an 'other' option in a multiple-choice question. Defaults to false.\"\n }\n },\n \"required\": [\n \"value\",\n \"label\"\n ],\n \"additionalProperties\": true\n },\n \"minItems\": 2,\n \"description\": \"Available choices for questions with a questionType of multiple_choice.\"\n }\n },\n \"required\": [\n \"minRequired\",\n \"maxAllowed\"\n ],\n \"additionalProperties\": true,\n \"description\": \"The choices for multiple_choice questions. Should not be defined for short_answer questions.\"\n }\n },\n \"required\": [\n \"questionId\",\n \"questionText\",\n \"questionType\"\n ],\n \"additionalProperties\": true,\n \"$schema\": \"http://json-schema.org/draft-07/schema#\"\n}",
"exception.stacktrace": "GenkitError: INVALID_ARGUMENT: Schema validation failed. Parse Errors:\n\n- questionType: must be equal to one of the allowed values\n\nProvided data:\n\n{\n \"questionId\": \"1\",\n \"questionText\": \"Is it man-made?\",\n \"questionType\": \"boolean\"\n}\n\nRequired JSON schema:\n\n{\n \"type\": \"object\",\n \"properties\": {\n \"questionId\": {\n \"type\": \"string\",\n \"description\": \"Unique identifier for this question. Used to match responses. Must be unique across all questions within a session or interaction.\"\n },\n \"questionText\": {\n \"type\": \"string\",\n \"description\": \"The text of the question presented to the user. Should be clear and concise.\"\n },\n \"questionType\": {\n \"type\": \"string\",\n \"enum\": [\n \"multiple_choice\",\n \"short_answer\"\n ],\n \"description\": \"The type of question. Determines the expected format of the response. Yes/No questions are defined as multiple_choice questions with only two choices, with a minAllowed and maxAllowed of one.\"\n },\n \"choices\": {\n \"type\": \"object\",\n \"properties\": {\n \"minRequired\": {\n \"type\": \"number\",\n \"description\": \"The minimum number of answers required. Must be an integer between 1 and the number of choices, inclusive.\"\n },\n \"maxAllowed\": {\n \"type\": \"number\",\n \"description\": \"The maximum number of answers allowed. Must be an integer between 1 and the number of choices, inclusive. Must be greater than or equal to minRequired.\"\n },\n \"choices\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"value\": {\n \"type\": \"string\",\n \"description\": \"The value representing the choice (e.g., 'a', '1', 'option_one'). This value is used in the client's response to identify the selected choice(s). Should be concise and easily machine-readable.\"\n },\n \"label\": {\n \"type\": \"string\",\n \"description\": \"The human-readable label for the choice (e.g., 'Option One', 'Red', 'United States'). This is what's displayed to the user.\"\n },\n \"allowTextEntry\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"Whether or not there should be a text input associated with this choice. Typically used for an 'other' option in a multiple-choice question. Defaults to false.\"\n }\n },\n \"required\": [\n \"value\",\n \"label\"\n ],\n \"additionalProperties\": true\n },\n \"minItems\": 2,\n \"description\": \"Available choices for questions with a questionType of multiple_choice.\"\n }\n },\n \"required\": [\n \"minRequired\",\n \"maxAllowed\"\n ],\n \"additionalProperties\": true,\n \"description\": \"The choices for multiple_choice questions. Should not be defined for short_answer questions.\"\n }\n },\n \"required\": [\n \"questionId\",\n \"questionText\",\n \"questionType\"\n ],\n \"additionalProperties\": true,\n \"$schema\": \"http://json-schema.org/draft-07/schema#\"\n}\n at parseSchema (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/core/src/schema.ts:133:21)\n at AsyncFunction.actionFn.run (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/core/src/action.ts:282:13)\n at actionFn (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/core/src/action.ts:266:28)\n at resolveToolRequest (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/resolve-tool-requests.ts:118:26)\n at <anonymous> (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/resolve-tool-requests.ts:174:55)\n at Array.map (<anonymous>)\n at resolveToolRequests (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/resolve-tool-requests.ts:171:33)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at generate (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/action.ts:336:5)\n at <anonymous> (/usr/local/google/home/gspencer/code/hil_test/node_modules/@genkit-ai/ai/src/generate/action.ts:130:22)"
},
"description": "exception"
}
}
]
}
}
}
}
If you run it again with includeSchema set to true, then the tool is called correctly.
Expected behavior
I would expect the schema definition for the tool input parameters to be given to the LLM automatically.
Runtime (please complete the following information):
OS: Linux
Version: 1.0.4
** Node version
v18.20.1
The text was updated successfully, but these errors were encountered:
Describe the bug
When calling a tool with a defined input schema, the LLM ignores this schema and makes up its own. If I include the schema in the prompt, then tool calling works as expected. It seems as thought the LLM isn't being shown the schema for the tool.
To Reproduce
Unpack this tar file:
twenty_questions.tar.gz
npx genkit start -- npx tsx src/index.ts
)twentyQuestionsFlow
with the following input where "includeSchema" is false:Input JSON
When you run it, you will see that you get errors similar to this trace:
Error Trace
includeSchema
set to true, then the tool is called correctly.Expected behavior
I would expect the schema definition for the tool input parameters to be given to the LLM automatically.
Runtime (please complete the following information):
** Node version
The text was updated successfully, but these errors were encountered: