Encountering API Call Error When Switching Provider from OpenAI to Azure OpenAI

[savbiz]

Hello everyone,

I'm in the process of transitioning the Promptfoo provider from OpenAI to Azure OpenAI, as outlined in the official documentation. However, despite adhering closely to the specified steps, I've encountered an issue during the API calls to the Azure OpenAI service.

Error encountered:

[FAIL] API call error: Error: Request failed after 4 retries: request to https://undefined/openai/deployments/gpt-35-turbo-16k/chat/completions?api-version=2023-12-01-preview failed, reason: getaddrinfo ENOTFOUND undefined

Steps I've taken:

I meticulously followed the guide to change the API provider settings from OpenAI to Azure OpenAI, setting the environment variable in my terminal as export AZURE_OPENAI_API_KEY={key}, where {key} is the value obtained from KEY 2 under "Keys and Endpoint" in the "Resource Management" section of the Azure dashboard.
Executed the evaluation tests using promptfoo eval --no-cache from the tests directory.

Using the latest version 0.50.0 and nvn 20.10.

The error message above was the result of these steps. It seems that the provider.config.apiHost is not read for some reason.

Test case setup:

prompts: [../../prompts/general_promptjson]
providers: [azureopenai:chat:gpt-35-turbo-16k]
defaultTest:
  options:
    provider:
      id: azureopenai:chat:gpt-35-turbo-16k
      config:
        apiHost: 'ai-qa-svc.openai.azure.com'
tests:
  - description: Conversation?
    vars:
      user_request: who are you?
    assert:
      - type: similar
      threshold: 0.8
      value: I'm an AI assistant.

Any insights or guidance on how to resolve this error would be greatly appreciated.

[Ian Webster]

Can you try setting it up so that you provide the apiHost in the providers list:

prompts: [../../prompts/general_promptjson]
providers: 
  - id: azureopenai:chat:gpt-35-turbo-16k
    config:
      apiHost: 'ai-qa-svc.openai.azure.com'

defaultTest:
  options:
    provider:
      # This provider is used for grading only. You'll have to switch this to an embeddings model if you want to use it for the `similar` assertion.
      id: azureopenai:chat:gpt-35-turbo-16k
      config:
        apiHost: 'ai-qa-svc.openai.azure.com'

tests:
  - description: Conversation?
    vars:
      user_request: who are you?
    assert:
      - type: similar
      threshold: 0.8
      value: I'm an AI assistant.

[savbiz]

That did it, I was indeed missing the config in the providers list.
Thanks @typpo! 🙌