OpenGenerativeAI / llm-colosseum

Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM

Home Page:https://huggingface.co/spaces/junior-labs/llm-colosseum

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

NEW DOCKER IMAGE ERROR: fighter starts, but two Kens do not fight.

taozhiyuai opened this issue Β· comments

fighter starts, but two Kens do not fight.
@oulianov
'
(0)Restarting system and (optionally) setting difficulty
🏟️ (864b) (0)Starting game
🏟️ (864b) (0)Waiting for fight to start
Exception in thread Thread-5:
Exception in thread Thread-6:
Traceback (most recent call last):
File "/Users/taozhiyu/miniconda3/envs/streetfighter/lib/python3.9/threading.py", line 980, in _bootstrap_inner
Traceback (most recent call last):
File "/Users/taozhiyu/miniconda3/envs/streetfighter/lib/python3.9/threading.py", line 980, in _bootstrap_inner
self.run()
File "/Users/taozhiyu/Downloads/llm-colosseum/eval/game.py", line 369, in run
self.run()
File "/Users/taozhiyu/Downloads/llm-colosseum/eval/game.py", line 383, in run
self.game.player_2.robot.plan()
File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 134, in plan
self.game.player_1.robot.plan()
File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 134, in plan
next_steps_from_llm = self.get_moves_from_llm()
File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 293, in get_moves_from_llm
next_steps_from_llm = self.get_moves_from_llm()
File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 293, in get_moves_from_llm
llm_stream = self.call_llm()
File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 369, in call_llm
llm_stream = self.call_llm()
File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 369, in call_llm
resp = client.stream_chat(messages)
AttributeError: 'NoneType' object has no attribute 'stream_chat'
resp = client.stream_chat(messages)
AttributeError: 'NoneType' object has no attribute 'stream_chat'
'

  • What model did you use ?
  • Are you using the latest version from main ?

This sounds like a duplicate issue of #50

latest version

    player_1=Player1(  
        nickname="Baby",  
        model="ollama:gemma:7b-instruct-q8_0",
    ),
    player_2=Player2(  
        nickname="Daddy",  
        model="ollama:llama3:8b-instruct-q8_0",

before I upgrade docker image, it works fine.

Done!

commented

hi @taozhiyuai , I encountered the same issues but using Mistral AI's API. Can you share how you resolved this?