bentoml / OpenLLM

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

Home Page:https://bentoml.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

bug: block GPU

estuday opened this issue · comments

Describe the bug

I find that using openllm will lock GPU computing resources. How can I unlock or reduce the locked resources

To reproduce

No response

Logs

No response

Environment

none

System information (Optional)

No response

i also find another question,after the openllm start command is run, if you press ctrl+c to end the process, port 3000 is always occupied and needs to be released manually