-
Notifications
You must be signed in to change notification settings - Fork 6
Features
Load balancing in the context of the OpenAI Hub is about distributing API requests across multiple API keys. This feature is designed to efficiently utilize available API keys, preventing overuse or depletion of any single key. The hub manages multiple API keys and automatically directs incoming requests to the key with the least load. This ensures a smoother and more stable operation, especially during peak times or high-traffic scenarios. It also helps to stay within the rate limits set by OpenAI for each API key.
API Key Protection is a security feature of the OpenAI Hub. Instead of requiring each user to have their own OpenAI API key, the hub allows users to make requests without needing an individual key. The hub manages all interactions with the OpenAI API, using its pool of API keys. This approach prevents API keys from being exposed to end users, enhancing security. It also simplifies the user experience, as users don't need to obtain and manage their own OpenAI API keys.
Global ACL (Access Control List) is a feature that provides control over which APIs and models users can access. This ensures that users can only access the resources they are supposed to. For example, you may wish to restrict certain users to only using specific models, or to limit their ability to make certain types of requests. The global ACL allows you to define these permissions at a high level, and applies them across all users and requests.
JWT Authentication is a secure and reliable user authentication system using JSON Web Tokens (JWT). You can issues a JWT for user via the cli tool openai-hub-jwt-token-gen, which the user then provides with each subsequent request. The hub verifies the JWT to authenticate the user's identity. This system provides a secure, stateless method of user authentication. It also supports additional features such as user roles and permissions, which can be encoded into the JWT.
The Access Log feature is a newly implemented function that allows tracking of API usage and token consumption. It records details of each request, including which user made the request, which API and model were used, and how many tokens were consumed. This information is valuable for monitoring and auditing purposes. The logs can be stored in several formats and backends, including file, SQLite, MySQL, or PostgreSQL, providing flexibility to suit different deployment environments and needs.