Giters

jay-mahadeokar / pynetbuilder

pyNetBuilder is a modular pytonic interface with builtin modules for generating popular caffe prototxt network file definitions.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Should ResNet work without Batch Norm layer?

mrgloom opened this issue · comments

commented

Should ResNet work without Batch Norm layer?

commented

Not sure it can. BatchNorm layer gives you a lot of advantages, including regularization, solves problem of internal covariate shift and diminishing gradients which are very important once you go deeper. It also helps to train faster, with bigger learning rates, which is necessary when you are training such deep networks. More details are presented in Batchnorm paper.