Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Google Gemini bug with finish_reason #281

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

mrmichaeladavis
Copy link

This continues with function/tool calling even if finish_reason == "stop".

Details:
I am using litellm OpenAI proxy to connect to none OpenAI models with extended_openai_conversation.

It is known that Google Gemini models respond with finish_reason=="stop" even if wanting to execute a tool or function. See BerriAI/litellm#2195 and BerriAI/litellm#6104

It appears more often with non-streaming.

Here is a log showing example response that causes the problem.

I tested and this happens with every Gemini model from 1.5 to latest 2.0 Flash.

2025-01-01 17:38:29.608 INFO (MainThread) [custom_components.extended_openai_conversation] Response {"id": "chatcmpl-ceb05157-2bb6-49d2-a43a-2cf2d0b81db4", "choices": [{"finish_reason": "stop", "index": 0, "message": {"role": "assistant", "tool_calls": [{"id": "call_64f0921a-b3b2-414a-b64f-095140cfd544", "function": {"arguments": "{\"list\": [{\"service\": \"close_cover\", \"domain\": \"cover\", \"service_data\": {\"entity_id\": \"cover.patio_door_shade\"}}]}", "name": "execute_services"}, "type": "function", "index": 0}]}}], "created": 1735774708, "model": "gemini-2.0-flash-exp", "object": "chat.completion", "usage": {"completion_tokens": 23, "prompt_tokens": 7557, "total_tokens": 7580}, "vertex_ai_grounding_metadata": [], "vertex_ai_safety_results": [[{"category": "HARM_CATEGORY_HATE_SPEECH", "probability": "NEGLIGIBLE"}, {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "probability": "NEGLIGIBLE"}, {"category": "HARM_CATEGORY_HARASSMENT", "probability": "NEGLIGIBLE"}, {"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "probability": "NEGLIGIBLE"}]], "vertex_ai_citation_metadata": []}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant