Feat: Support vLLM with OpenAI compatible chat API

[Sam Stoelinga]

Currently vLLM will throw the following error when Verba send a chat completion request:

✘ WebSocket Error: Invalid response object from API:
'{"object":"error","message":"Conversation roles must alternate
user/assistant/user/assistant/...","type":"BadRequestError","param":null,"code":400}'
(HTTP response code was 400)

I believe this is due to Verba not correctly alternating between user/assistant/user as the error message indicates.

Steps to reproduce:

1. Deploy vLLM with openai compatible endpoint and mistral v2 7b
2. Set OPENAI_BASE_URL=http://vlm-end-point/v1

[Sam Stoelinga]

Maybe we should fix it on the vLLM side by doing something like this: vllm-project/vllm#2112 (comment)

[Sam Stoelinga]

Closing this since I was able to fix it by using a custom chat template for Mistral with vLLM