Will future support include expert parallel and sequence parallel (such as ring attention)?
Doraemonzzz opened this issue · comments
Great job! Will future support include expert parallel and sequence parallel (such as ring attention)?
Yes we are actively working on them, so stay tuned :)