How can I use Multiple NPUs ?

Question

QuentinWang1 opened this issue 2 months ago · comments

for example :
python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5 --num-gpus 2

maybe this following command need to be supported?
python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5 --device npu --num-npus 2