bentoml / OpenLLM

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

Home Page:https://bentoml.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

I'm having trouble getting statted with openllm, but I don't want to use conda and I have WSL2

Lightwave234 opened this issue · comments

All I want to do is setup a simple openllm using the Windows Subsystem for Linux, but I keep running into issues getting a response from the the thing, it's also worth noting that I'm using Ubuntu 22.04 and Python3.12 to run the thing, IDK if that has any application to the functionality of the app, but I'm open to suggestions if anyone has any.

commented

Try python 3.11. I've had problems with python 3.12 as well.