Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix cohere chat blueprint #2167

Merged
merged 4 commits into from
Feb 29, 2024
Merged

fix cohere chat blueprint #2167

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a1afd25
fix cohere chat blueprint
ylwu-amzn Feb 29, 2024
05bf8c8
remove empty line
ylwu-amzn Feb 29, 2024
f3ce30b
remove extra comma
ylwu-amzn Feb 29, 2024
a7558cd
update response
ylwu-amzn Feb 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 39 additions & 25 deletions docs/remote_inference_blueprints/cohere_connector_chat_blueprint.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ See [Cohere's /chat API docs](https://docs.cohere.com/reference/chat) for more d

```json
POST /_plugins/_ml/model_groups/_register

{
"name": "cohere_model_group",
"description": "Your Cohere model group"
Expand Down Expand Up @@ -56,7 +55,7 @@ POST /_plugins/_ml/connectors/_create
"Authorization": "Bearer ${credential.cohere_key}",
"Request-Source": "unspecified:opensearch"
},
"request_body": "{ \"message\": ${parameters.message}, \"model\": \"${parameters.model}\" }"
"request_body": "{ \"message\": \"${parameters.message}\", \"model\": \"${parameters.model}\" }"
}
]
}
Expand All @@ -70,7 +69,6 @@ You will now register the model you created using the `model_group_id` and `conn

```json
POST /_plugins/_ml/models/_register

{
"name": "Cohere Chat Model",
"function_name": "remote",
Expand Down Expand Up @@ -126,7 +124,7 @@ Note: `conversation_id` here is the string value for "1", but you can use any st
POST /_plugins/_ml/models/<MODEL_ID_HERE>/_predict
{
"parameters": {
"message": "What is the weather like in London?",
"message": "What is the weather like in London?"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this , the response is

{
  "inference_results": [
    {
      "output": [
        {
          "name": "response",
          "dataAsMap": {
            "response_id": "b0f6eaf6-3f63-418b-9178-a5db8431e55e",
            "text": """Here is the current weather and forecast for London, UK:

**Current Weather:**

- Temperature: 26.0°C (79°F)
- Conditions: Cloudy with occasional sunny spells
- Humidity: 62%
- Wind: 3 mph (5 km/h) (NW)
- Visibility: 10 km (6 miles)

**Forecast for the next few days:**

The weather in London will continue to be moderately warm for the next few days with temperatures reaching 27°C (81°F) on Saturday. The coming days will be mostly cloudy with occasional sunny spells and chances of light rain throughout the weekend. 

Make sure to carry an umbrella if you are going outside, and consider using sunscreen as and when required. 

If you would like a more detailed or specific weather forecast for London, let me know, and I'll be happy to provide it!""",
            "generation_id": "89c59a14-f099-4100-95b0-7d0d68104f4b",
            "finish_reason": "COMPLETE",
            "token_count": {
              "prompt_tokens": 70,
              "response_tokens": 189,
              "total_tokens": 259,
              "billed_tokens": 248
            },
            "meta": {
              "api_version": {
                "version": "1"
              },
              "billed_units": {
                "input_tokens": 59,
                "output_tokens": 189
              }
            }
          }
        }
      ],
      "status_code": 200
    }
  ]
}

The result is different with the blueprint, it doesn't have

"documents": [
                ...
            ],
            "search_results": [
                ...
            ],
            "tool_inputs": null,
            "search_queries": [
                ...
            ]

Is that correct , @tianjing-li ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks good, thank you!

}
}
```
Expand All @@ -139,31 +137,47 @@ It should return a response similar to this:
{
"output": [
{
"response_id": "f92fdef6-e43c-465f-a2b8-45772b9ef39d",
"text": "The weather on Thursday, February 1, 2018, in London will be an overcast high of 13°C and a low of 10°C. Unfortunately, I cannot give you a detailed weather forecast for the next ten days in London, as it varies considerably across different sources. Would you like to know more about the weather on any particular day within the next ten?",
"generation_id": "76e5c68c-a3ca-40a0-91a9-20315f52b4c4",
"name": "response",
"dataAsMap": {
"response_id": "b0f6eaf6-3f63-418b-9178-a5db8431e55e",
"text": """Here is the current weather and forecast for London, UK:

**Current Weather:**

- Temperature: 26.0°C (79°F)
- Conditions: Cloudy with occasional sunny spells
- Humidity: 62%
- Wind: 3 mph (5 km/h) (NW)
- Visibility: 10 km (6 miles)

**Forecast for the next few days:**

The weather in London will continue to be moderately warm for the next few days with temperatures reaching 27°C (81°F) on Saturday. The coming days will be mostly cloudy with occasional sunny spells and chances of light rain throughout the weekend.

Make sure to carry an umbrella if you are going outside, and consider using sunscreen as and when required.

If you would like a more detailed or specific weather forecast for London, let me know, and I'll be happy to provide it!""",
"generation_id": "89c59a14-f099-4100-95b0-7d0d68104f4b",
"finish_reason": "COMPLETE",
"token_count": {
"prompt_tokens": 1523,
"response_tokens": 74,
"total_tokens": 1597,
"billed_tokens": 81
"prompt_tokens": 70,
"response_tokens": 189,
"total_tokens": 259,
"billed_tokens": 248
},
"meta": {
...
},

"documents": [
...
],
"search_results": [
...
],
"tool_inputs": null,
"search_queries": [
...
]
"api_version": {
"version": "1"
},
"billed_units": {
"input_tokens": 59,
"output_tokens": 189
}
}
}
}
]
],
"status_code": 200
}
]
}
Expand Down
Loading