Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: removes AWS "us." region prefix from model id #10329

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

gtang31-te
Copy link

@gtang31-te gtang31-te commented Jan 28, 2025

Motivation and Context

relates to this issue.

Description

The current implementation of bedrock_model_provider.py is hard coded to return the zeroth element when splitting model_id on (.) periods. However this will return an incorrect name for the provider when working with an inference profile since the model_id will contain the AWS region in the ID. For example, a model_id can be us.meta.llama3-XXX:0 and the call to_model_provider will incorrectly try to look for a us model. This error is thrown when we are selecting an LLM model using an AWS inference profile.

I have made a change to the to_model_provider class method in the bedrock_model_provider.py script that checks for a region prefix and then strips it. I have not tested this against all AWS regions.

Contribution Checklist

@gtang31-te gtang31-te requested a review from a team as a code owner January 28, 2025 19:22
@markwallace-microsoft markwallace-microsoft added the python Pull requests for the Python Semantic Kernel label Jan 28, 2025
@gtang31-te gtang31-te force-pushed the python-removes-aws-region-prefix-from-model-name branch from 3903914 to 80de5d0 Compare February 1, 2025 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python Pull requests for the Python Semantic Kernel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants