Giters

yxuansu / PandaGPT

[TLLM'23] PandaGPT: One Model To Instruction-Follow Them All

Home Page:https://panda-gpt.github.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

About model size

David-Zeng-Zijian opened this issue · comments

commented

Excellent work! I just wonder if there is any way to load the model in more than one GPU because even the 7B model consumes more than 20G memory, which is larger than memory of one GPU.

commented

Hi @David-Zeng-Zijian, the current model requires around 24GB memory to run on a single GPU. Some possible solutions include model quantization and model parallel. It would be great if you can share us some of your solutions to deploy PandaGPT across multiple GPUs.