bentoml / OpenLLM

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

Home Page:https://bentoml.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

feat: any plan to support NPU

zhangxinyang97 opened this issue · comments

Feature request

support Ascend NPU

Motivation

more and more developer using NPU, and vllm is also interesting in NPU ecosystem, vllm-project/vllm#1606.

Other

No response

we don't have any immediate plan atm.