why i use vllm inference deepseek v2 ,speed is low

[ZzzybEric]

i use vllm to inference deepspeed, use flask to deploy model. When the problem enters the model, it always gets stuck for a long time in the processd prompt step，the code i use is your example code

[Fuli Luo]

https://huggingface.co/deepseek-ai/DeepSeek-V2/discussions/1
@ZzzybEric

[ran130683]

whats your gpu type？