Aquila2推理优化：请问Aquila2-34b在量化后还想使用单机多卡和多机多卡并行推理以提高效率，该如何实现呢？
monkeyZhy opened this issue · comments
Description
请问Aquila2-34b在量化后还想使用单机多卡和多机多卡并行推理以提高效率，该如何实现呢？
Alternatives
No response
FlagAI-Open / FlagAI
FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.