deep_learning_normalization
A personal pet project to explore different kinds of normalizations in Deep Learning research
Papers we'll be working with -
- Batch Renormalization: https://arxiv.org/pdf/1702.03275.pdf
- Group Normalization: https://arxiv.org/pdf/1803.08494.pdf
- Batch Instance Norm: https://arxiv.org/pdf/1805.07925.pdf
- Layer Normalization: https://arxiv.org/pdf/1607.06450.pdf
- Instance Normalization: https://arxiv.org/pdf/1607.08022.pdf
- Kalman Normalization: https://papers.nips.cc/paper/7288-kalman-normalization-normalizing-internal-representations-across-network-layers.pdf
- Batch Normalization
If I missed something, please suggest and I'll add them.
Goal:
- Read all papers, upload annotated versions for others to read.
- Write a summary of all of them.
- Code them up and compare on a task - Classification?