Coursework of Deep Learning in 2021
[Team Work]
Jiawen -- trajectory subsampling
Chuang -- quantifying separation
Zhaoxing -- gini feature importance analysis
Jiawei -- noise robustness
Team members conducted experiments, analyzed the results and wrote corresponding reports.
[Abstract]
The paper introduced a virtual experiment to infer the learning algorithm which modifies the parameters of a neural network, from the trajectory it induces. A number of different deep artificial neural network models are trained on ImageNet tasks with four learning algorithms. While learning proceeds, the authors collect aggregate statistics. These aggregate quantities are then used as predictors to train simple classifiers, which attempt to discriminate which of the four learning rules trained the model. We divide the authors' work into four parts and verify the results separately: quantifying separation, gini feature importance analysis, trajectory subsampling, noise robustness. We have partially successfully reproduced the relevant experiments in the original paper and confirmed some of the conclusions of the original paper.
The paper's link: https://proceedings.neurips.cc/paper/2020/file/1ba922ac006a8e5f2b123684c2f4d65f-Paper.pdf