yjchun626 / azure-openai-sdk-proxy

This provides a proxy server application of Azure OpenAI Service API that round-robins multiple Azure OpenAI Service instances.

Azure OpenAI Service Proxy

This provides a proxy server application of Azure OpenAI Service API that round-robins multiple Azure OpenAI Service instances.

Prerequisites

• Azure Subscription
• Azure OpenAI Subscription
• .NET SDK 8.0.300+ + Aspire workload
• Visual Studio 2022 17.10+ or Visual Studio Code + C# DevKit
• Docker Desktop or Podman
• PowerShell 7.4+
• Azure Developer CLI
• Azure CLI + Container Apps extension
• GitHub CLI

Getting Started

1. Login to Azure and GitHub.

   # Login to Azure Dev CLI
   azd auth login
   
   # Login to Azure CLI
   az login
   
   # Login to GitHub
   gh auth login

2. Check login status.

   # Azure Dev CLI
   azd auth login --check-status
   
   # Azure CLI
   az account show
   
   # GitHub CLI
   gh auth status

3. Fork this repository to your account and clone the forked repository to your local machine.

   gh repo fork aliencube/azure-openai-sdk-proxy --clone --default-branch-only

4. Run the following commands in order to provision and deploy the app.

   # zsh/bash
   AZURE_ENV_NAME="proxy$((RANDOM%9000+1000))"
   azd init -e $AZURE_ENV_NAME
   azd up
   
   # PowerShell
   $AZURE_ENV_NAME = "proxy$(Get-Random -Minimum 1000 -Maximum 9999)"
   azd init -e $AZURE_ENV_NAME
   azd up

   NOTE: The AZURE_ENV_NAME variable is an arbitrary name for the Azure environment. You can change it to your preferred one.

5. Run the following command to provision Azure resources that are required for the app.

   # zsh/bash
   AZURE_RESOURCE_GROUP="rg-$AZURE_ENV_NAME"
   AZURE_LOCATION=$(azd env get-value AZURE_LOCATION)
   az deployment group create \
       -g $AZURE_RESOURCE_GROUP \
       --template-file ./infra/aspire.bicep \
       --parameters environmentName=$AZURE_ENV_NAME \
       --parameters location=$AZURE_LOCATION
   
   # PowerShell
   $AZURE_RESOURCE_GROUP = "rg-$AZURE_ENV_NAME"
   $AZURE_LOCATION = azd env get-value AZURE_LOCATION
   az deployment group create `
       -g $AZURE_RESOURCE_GROUP `
       --template-file ./infra/aspire.bicep `
       --parameters environmentName=$AZURE_ENV_NAME `
       --parameters location=$AZURE_LOCATION

Unit Testing & Integration Testing & Linting OpenAPI Document

1. Run the following command to run both unit tests and integration tests at once on your local machine.

   # zsh/bash
   ./scripts/run-tests.sh -c Debug -k || ./scripts/run-tests.sh -c Debug
   
   # PowerShell
   ./scripts/Run-Tests.ps1 -Configuration Debug -KillPorts