why use the class LayerNorm()? why not using the nn.LayerNorm(dim) in pytorch?
henbucuoshanghai opened this issue · comments
henbucuoshanghai commented
class LayerNorm(nn.Module):
r""" LayerNorm that supports two data formats: channels_last (default) or channels_first.
The ordering of the dimensions in the inputs. channels_last corresponds to inputs with
Zhuang Liu commented
It's because nn.LayerNorm only supports the case when the channel dimension is at the last. Sometimes our data tensor is NCHW, not NHWC.