confident-ai / deepeval

The LLM Evaluation Framework

Home Page:https://docs.confident-ai.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add Cohere command-r model support

fabian57fabian opened this issue · comments

❗BEFORE YOU BEGIN❗
Are you on discord? 🤗 We'd love to have you asking questions on discord instead: https://discord.com/invite/a3K9c8GRGt

Is your feature request related to a problem? Please describe.
Allow users to use Cohere api with command-r model.

Describe the solution you'd like
Create a CohereModel class from DeepEvalBaseLLM according to https://docs.confident-ai.com/docs/metrics-introduction#using-a-custom-llm

Describe alternatives you've considered
Creating a personal class and copying between projects can be struggling.

Additional context
Cohere offers other models and one can experiment with them.

Tests are not running because poetry does not install cohere.
Any idea how to change the poetry.lock file?

They now also released a larger, 104B parameter model: C4AI Command R+

They now also released a larger, 104B parameter model: C4AI Command R+

I added a pull requests by inserting the class in the docs. Still waiting for reply.
This relies entirely on the cohere api, but i don't know if they allow what model to use or if the api key decides the model used.
Check out #661 PR

just merged, TYSM! Would love to have you in our discord, especially when I start merging your docs into some tutorials for our users: https://discord.com/invite/a3K9c8GRGt

Tests are not running because poetry does not install cohere. Any idea how to change the poetry.lock file?

(dw about this, its fine)