New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

多渠道都配置了 GPT4o 模型，是否可以负载均衡 #2033

Open

1 task done

akayj opened this issue Jan 14, 2025 · 0 comments

Open

1 task done

多渠道都配置了 GPT4o 模型，是否可以负载均衡 #2033

akayj opened this issue Jan 14, 2025 · 0 comments

Labels

bug

akayj commented Jan 14, 2025 •

edited

Loading

例行检查

[ x] 我已确认目前没有类似 issue
[ x] 我已确认我已升级到最新版本
[x ] 我已完整查看过项目 README，尤其是常见问题部分
[ x] 我理解并愿意跟进此 issue，协助测试和提供反馈
我理解并认可上述内容，并理解项目维护者精力有限，不遵循规则的 issue 可能会被无视或直接关闭

问题描述
配置了多个渠道，每个渠道都是不同的Azure GPT4o deployment, 客户端请求一直路由到其中一个，并没有负载均衡

复现步骤

预期结果
可以将不同的 azure deployment 的同样的 model，作为一个集群的概念，客户端访问后，负载到其中一个。

相关截图

akayj added the bug label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment