OpenAI API Proxy is a transparent middleware service built using Python and FastAPI, designed to sit between clients and the OpenAI API. The proxy supports all models and APIs of OpenAI, streams OpenAI's responses back to clients in real-time, and logs request timestamps, response times, status codes, request contents and response contents to a database for future querying and maintenance.
Note that this program was originally implemented in collaboration with GPT-4.
- Transparent Proxy: Supports all OpenAI models and APIs. Request paths, models, and OPENAI_API_KEY are all obtained from the original requests.
- Streaming: Streams OpenAI's responses back to clients in real-time.
- Logging: Records request timestamps, response times, status codes, request contents, response contents (if successful), and institution IDs to a database.
- Error Handling: Handles request failures.
- Enterprises and Institutions: Establish a secure middleware layer between clients and the OpenAI API for internal monitoring and management of API usage.
- Data Analysis: Collect and analyze log data to understand hotspots and trends in API requests, optimizing API usage and performance.
- Billing and Quota Management: Track API requests per institution to implement usage-based billing and quota management.
- Auditing and Security: Ensure API requests comply with enterprise and institutional security policies and assist in auditing and tracking.
- Performance Monitoring: Monitor API response times and status codes in real-time to identify and resolve potential performance issues.
- API Version Control and Migration: Facilitate smooth API version migration by handling version discrepancies at the proxy layer, reducing client migration costs.
- Install dependencies:
pip install -r requirements.txt- Run the proxy server:
uvicorn main:app --host 127.0.0.1 --port 8000- Make requests to the proxy server, specifying the OpenAI API endpoint, model, and any other required parameters:
curl -X POST "http://localhost:8000/v1/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <your_openai_api_key>" \
-d '{"model": "text-davinci-003", "prompt": "Once upon a time...", "max_tokens": 100, "temperature": 0}'Replace <your_openai_api_key> with your actual OpenAI API key.
In summary, OpenAI API Proxy is a flexible and powerful tool designed to help enterprises and institutions better manage and monitor their access to the OpenAI API, improving security, control, and performance.
This program is licensed under the MIT License. Please see the LICENSE file for details.