vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Home Page:https://docs.vllm.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature]: Host CPU Docker image on Docker Hub

VMinB12 opened this issue Β· comments

πŸš€ The feature, motivation and pitch

Currently vllm has pre-built docker images for vllm/vllm-openai. This image requires a GPU to run. Now that CPU support is being added, it would be great if CPU images could be added too.
Allowing users to pull CPU images could greatly simplify the process for users and relaxing the GPU requirement would allow more users to benefit.

Alternatives

No response

Additional context

No response