Giters

hpcaitech / EnergonAI

Large-scale model inference.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to inference BLOOM-176B by multi-node multi-card？

vicwer opened this issue · comments

commented

HI，Thank you very much for your work, I can now do bloom-560m inference on single node 4 cards by example, how to use multi-node multi-card (like 4x4x32GB v100GPU) inference bloom-176b？

commented

same asking here

commented

me too! how to use multi-node multi-card inference? Does Energonai support ?