Add classical/new architectures
omarsar opened this issue · comments
Some ideas for figures to add to the PPT
- Linear regression, single-layer neural network
- Multilayer Perceptron with hidden layer
- Backpropagation
- Batch Normalization and alternatives
- Computational Graphs
- Dropout
- CNN - padding, stride, pooling,...
- LeNet
- AlexNet
- VGG
- GoogleNet
- ResNet
- DenseNet
- Memory Networks
- RNN
- Deep RNN
- Bidirectional RNN
- GRU
- LSTM
- Language RNN models
- Backpropagation through time
- Encoder-Decoder Architecture
- Seq2seq with RNN encoder-decoder
- Bearm search and other decoding strategies
- Attention
- Multi-head attention
- Self-attention
- Transformer
- Word2vec/GloVe/Skip-gram/CBOW/BERT/GPT....
- Common/Popular CV/NLP Tasks
List adopted from multiple resources including nlpoverview and d2l.ai which both contain a very solid syllabus.
Please feel free to make suggestions below. If you would like to help, also let me know.
Maybe, the generative adversarial networks and graph neural networks should be included in the list.
Deep Belief Networks
Can we Graph Neural Network models like GCN, GAT, GGNN, GraphSage, etc?
Including Contrastive Representation Learning models would be great!