bentoml / OpenLLM

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

https://bentoml.com

feat: add vllm 0.3.0 support

whyiug opened this issue 4 months ago · comments

whyiug commented 4 months ago

Feature request

tag 0.3.0 is a big update, please support.
https://github.com/vllm-project/vllm/releases/tag/v0.3.0

Motivation

No response

Other

No response

Aaron Pham commented 4 months ago

Will work on it this week.

RLe commented 3 months ago

Linked pull request #925

Aaron Pham commented 3 months ago

This is the PR #821