Race condition when calculating the loss

[Luke Donghyun Lee]

Settings

• MNIST
• K5

With parallelization

n10000
Losses:  [79.48751831054688, 80.32427978515625, 77.17288208007812]

n30000
Losses:  [79.2096939086914, 76.24256896972656, 83.11274719238281]

Without parallelization

n10000
Losses:  [97.50489807128906, 96.83230590820312, 96.83230590820312]

n30000
Losses:  [96.84100341796875, 96.79830169677734, 96.79830169677734]