calculate BN on multi GPUs?
FightForCS opened this issue · comments
Hi, thanks for your great work. I have a question on BN calculation. In your code, BN is only calculated on a single GPU. What if I want calculate BN on multi GPUs to make the normalization step more accurate? Since it may help network converge better. Thanks.
BN is important. I think it should work better. But I don't have a good implementation of multi-gpu BN in tensorflow.
Thx