Fix TP embedding layers
jason9693 opened this issue · comments
Describe a TODO feature
- Force tp_wrapper do not parallelize emb-layer if model has not embedding layer. (for vision model competible)
https://discord.com/channels/729741769192767510/1012603449910759504/1083785802930192434