feat: bedrock #5
Conversation
["፧ Initializing Bedrock model."]
⛔ ERRORS
Error in generateText:
{}
⛔ ERRORS
ERROR:
{}
|
Nice! Let me know when it's ready to be tested, happy to support. |
|
https://github.com/meta-introspector/reppy/tree/feature/23 this is my
tester now to integrate
…On Fri, Jan 10, 2025, 14:19 emile baizel ***@***.***> wrote:
Nice! Let me know when it's ready to be tested, happy to support.
—
Reply to this email directly, view it on GitHub
<#5 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AD5KQ2L4YQ5XZILXAE7RSET2KAMMJAVCNFSM6AAAAABU6SQKHCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBTG4YDAMBTHA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
first you need to accept the license etc from aws anthropic etc.
this works only in region us west, and still does not work 100%
the error is
["፧ Initializing Bedrock model."]
⛔ ERRORS
Error in generateText:
{"name":"ValidationException","$fault":"client","$metadata":{"httpStatusCode":400,"requestId":"30292ec9-a8b8-40f9-b904-ca56140bf25f","attempts":1,"totalRetryDelay":0}}
⛔ ERRORS
ERROR:
{"name":"ValidationException","$fault":"client","$metadata":{"httpStatusCode":400,"requestId":"30292ec9-a8b8-40f9-b904-ca56140bf25f","attempts":1,"totalRetryDelay":0}}
["፧ Retrying..."]
so i think that might be the parameters.
export AWS_REGION=us-west-2
you need to requst access to the models in aws. not all the models are available everywhere, using us-west-2
https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/modelaccess
list model
`aws bedrock list-foundation-models`
install aws cli new ```
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install --bin-dir /usr/local/bin --install-dir /usr/local/aws-cli --update
```
```
eval $(/usr/local/aws-cli/v2/current/bin/aws configure export-credentials --profile default --format env)
```
|
With this new version it is accessing the model after you accept the terms it is getting a 400 error on the request but we can fix that, |
|
now to add the embeddings model, removing local embeddings for now. |
I am going to use the proxy api instead of porting this right now. |
|
Is Eliza built on the OpenAI APIs/SDKs? Running an EKS cluster is a pretty heavy lift, and I don't know if we can expect every developer who wants to use Bedrock to have to run a cluster. |
|
i will have a lightweight proxy server that runs locally https://github.com/jparkerweb/bedrock-wrapper here is another option in node |
|
currently the patched code I wrote needs to work on the embeddings, |
|
have you seen litellm? |
|
yes i have a fork of it i will be moving forward
jmikedupont2/openlightllm#4
…On Mon, Jan 13, 2025, 13:52 emile baizel ***@***.***> wrote:
have you seen litellm <https://github.com/BerriAI/litellm>?
—
Reply to this email directly, view it on GitHub
<#5 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AD5KQ2JDSY5HDGZOMUNVW432KQDQVAVCNFSM6AAAAABU6SQKHCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBXHE2DGMBTG4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
There was a problem hiding this comment.
hey, are you still planning to incorporate Bedrock? i wasn't clear from your comments about testing out google and use openai api.
the approach you have here aligns with how google was added as a provider
|
I will continue to work on it, when I have time. Just need to do the
decoder part. So I will deliver something, just finishing up the arm64 that
is taking longer. but this is not as much as a prioritiy for my right now
because i feel we can work around the bedrock for now. once i get the
server running with arm we can get bedrock running.
…On Tue, Jan 14, 2025, 11:22 emile baizel ***@***.***> wrote:
***@***.**** commented on this pull request.
hey, are you still planning to incorporate Bedrock? i wasn't clear from
your comments about testing out google and use openai api.
the approach you have here aligns with how google was added as a provider
<https://github.com/elizaOS/eliza/pull/246/files>
—
Reply to this email directly, view it on GitHub
<#5 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AD5KQ2IL4GYJJ7676VCIUE32KU2UTAVCNFSM6AAAAABU6SQKHCVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDKNJQGMYDIOJRGQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***
com>
|
|
I cannot work on two tasks with such complexity at once, this is going to have to wait, i tried to start the debugger but i need to learn to debug this |
adding bedrock to the aws branch for now...
needs to be tested and rebased.