Aquila2推理优化:请问Aquila2-34b在量化后还想使用单机多卡和多机多卡并行推理以提高效率,该如何实现呢?
monkeyZhy opened this issue · comments
Description
请问Aquila2-34b在量化后还想使用单机多卡和多机多卡并行推理以提高效率,该如何实现呢?
Alternatives
No response
FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.
monkeyZhy opened this issue · comments
请问Aquila2-34b在量化后还想使用单机多卡和多机多卡并行推理以提高效率,该如何实现呢?
No response