OpenBMB / ChatDev

Create Customized Software using Natural Language Idea (through LLM-powered Multi-Agent Collaboration)

Home Page:https://arxiv.org/abs/2307.07924

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add ollama integration instruction in the readme .

hemangjoshi37a opened this issue · comments

Please anyone who has write access to this repo please provide any documentation or anything on how to replace openAI models with ollama models . thanks .

commented

Hi:) You can adjust the relevant settings in the model_backend.py, including modeltype, your API_KEY, etc.

If you want to use Ollama in local setup, you can follow my steps:

  1. Set environment variables:
    export OPENAI_API_KEY=ollama  # any value
    export BASE_URL=http://localhost:11434/v1  # your Ollama API server
  2. Replace model parameter to your model in Ollama:
    response = client.chat.completions.create(*args, **kwargs, model=self.model_type.value,

    Example:
    response = client.chat.completions.create(*args, **kwargs, model="gemma:2b-instruct",
                                              **self.model_config_dict)
  3. Run:
    python3 run.py --task "[description_of_your_idea]" --name "[project_name]"

@thinh9e ok thanks. but this should be added to the readme file so that it is accessible to anyone.

If you want to use Ollama in local setup, you can follow my steps:

  1. Set environment variables:

    export OPENAI_API_KEY=ollama  # any value
    export BASE_URL=http://localhost:11434/v1  # your Ollama API server
  2. Replace model parameter to your model in Ollama:

    response = client.chat.completions.create(*args, **kwargs, model=self.model_type.value,

    Example:

    response = client.chat.completions.create(*args, **kwargs, model="gemma:2b-instruct",
                                              **self.model_config_dict)
  3. Run:

    python3 run.py --task "[description_of_your_idea]" --name "[project_name]"

Hi. I get the below error. it seems the model name has to be supported by tiktoken. Is there any way to bypass it so that open models can be used?