Giters

rashadphz / farfalle

🔍 AI search engine - self-host with local or cloud LLMs

Home Page:https://www.farfalle.dev/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Azure OpenAI API Support

crypticatul opened this issue · comments

commented

Hi,

I want to utilize my Azure OpenAI API for this, can you add and option for the same? GPT-3.5-Turbo, and GPT-4o will be enough as an option. I am deploying on Vercel.

Thanks.

commented

Unfortunately, at this time, all providers are hardcoded. It would be nice to have them support this through a config file. I got Azure working but had to add it in a lot of places in both the backend and frontend. Basically, it's pretty much like OpenAI but you have to use a different llama_index client. example in chat.py --> get_llm

...
elif model == ChatModel.AZURE:
    return AzureOpenAI(
            api_version=os.environ["AZURE_OPENAI_API_VERSION"],
            deployment_name=os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"],
            model=os.environ["AZURE_OPENAI_MODEL"],
            api_key=os.environ["AZURE_OPENAI_API_KEY"],
            azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
        ) 
...

Also you will have to implement your Instructor to support the related queries.

related_queries.py > instructor_client

...
    elif model == ChatModel.AZURE:
        return from_azureopenai(openai.AsyncAzureOpenAI(
            api_version=os.environ["AZURE_OPENAI_API_VERSION"],
            azure_deployment=os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"],
            api_key=os.environ["AZURE_OPENAI_API_KEY"],
            azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
        ))
...
commented

Hey, I added support for azure! Let me know if you have any trouble setting it up!

commented

Hi, thanks for adding the support for Azure. I just have a few silly queries;
In env vars

  • On Azure, I have deployed gpt-35-turbo-16k (model name) with the deployment name "Better" so what should I use in AZURE_DEPLOYMENT_NAME
  • What we need to put in AZURE_API_BASE? is it region name or Azure Endpoint Name? I have deployed this in eastus
  • For Azure the OPENAI_MODE=azure, right?
  • Apologies for silly queries, however, generally speaking, to use this deployment I just use Endpoint, API Key, Deployment Name and API Version, however, some fields in the env vars template are a bit confusing to me.

I'm using the below env vars and still getting error - OPENAI_API_KEY environment variable not found

image