minsoo9506 / cs224w-study

study about GNN

GNN study

• 기간: 2022.08.01 ~
• 내용: cs224w 강의

Road

Tutorial

• Code
  • networkx
  • torch_geometric

Node Embeddings

• Lecture
  • 3.1 - Node Embeddings
  • 3.2 - Random Walk Approaches for Node Embeddings
  • 3.3 - Embedding Entire Graphs
• Code
  • cs224w HW colab 1

PageRank

• Lecture
  • 4.1 - PageRank
  • 4.2 - PageRank: How to Solve?
  • 4.3 - Random Walk with Restarts
  • 4.4 - Matrix Factorization and Node Embeddings

Node Classificaiton

• Lecture
  • 5.1 - Message passing and Node Classification
  • 5.2 - Relational and Iterative Classification
  • 5.3 - Collective Classification

GNN

• Lecture
  • 6.1 - Introduction to Graph Neural Networks
  • 6.2 - Basics of Deep Learning
  • 6.3 - Deep Learning for Graphs
  • 7.1 - A general Perspective on GNNs
  • 7.2 - A Single Layer of a GNN
  • 7.3 - Stacking layers of a GNN
  • 8.1 - Graph Augmentation for GNNs
  • 8.2 - Training Graph Neural Networks
  • 8.3 - Setting up GNN Prediction Tasks
  • 9.1 - How Expressive are Graph Neural Networks
  • 9.2 - Designing the Most Powerful GNNs
• Code
  • cs224w HW colab 2
    • GCN
  • cs224w HW colab 3
    • GraphSAGE
    • GAT
• Summary
  • GCN
  • GraphSAGE
  • GAT
• Paper
  • Design Space of Graph Neural Networks, NIPS 2020
  • GCN: Semi-Supervised Classification with Graph Convolutional Networks, ICLR 2017
  • GraphSAGE: Inductive Representation Learning on Large Graphs, NIPS 2017
  • GAT: Graph Attention Networks, ICLR 2018
  • Batch Normalization: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, ICML 2015
  • Dropout: Dropout: A Simple Way to Prevent Neural Networks from Overfitting, JMLR 2014
  • Skip connection: Deep Residual Learning for Image Recognition
  • Skip connection in Graph: Representation Learning on Graphs with Jumping Knowledge Networks, ICML 2018
  • GIN: HOW POWERFUL ARE GRAPH NEURAL NETWORKS?, ICLR 2019

Heterogeneous Graphs, Knowledge Graph Embeddings

• Lecture
  • 10.1-Heterogeneous & Knowledge Graph Embedding
  • 10.2 - Knowledge Graph Completion
  • 10.3 - Knowledge Graph Completion Algorithms
  • 11.1 - Reasoning in Knowledge Graphs
  • 11.2 - Answering Predictive Queries
  • 11.3 - Query2box: Reasoning over KGs
• Paper
  • TranE: Translating Embeddings for Modeling Multi-relational Data
  • TranR: Learning Entity and Relation Embeddings for Knowledge Graph Completion
  • DistMult: Embedding Entities and Relations for Learning and Inference in Knowledge Bases
  • ComplEx: Complex Embeddings for Simple Link Prediction
  • Traversing Knowledge Graphs in Vector Space
  • Embedding Logical Queries on Knowledge Graphs
  • Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings

Advanced Topics in Graph Neural Networks

• Lecture
  • 16.1 - Limitations of Graph Neural Networks
  • 16.2 - Position-Aware Graph Neural Networks
  • 16.3 - Identity-Aware Graph Neural Networks
  • 16.4 - Robustness of Graph Neural Networks
• Paper
  • Position-aware Graph Neural Networks, ICML 2019
  • Identity-aware Graph Neural Networks, AAAI 2021
  • Adversarial Attacks on Neural Networks for Graph Data, KDD 2018

Scaling Up Graph Neural Networks to Large Graphs

• Lecture
  • 17.1 - Scaling up Graph Neural Networks
  • 17.2 - GraphSAGE Neighbor Sampling
  • 17.3 - Cluster GCN: Scaling up GNNs
  • 17.4 - Scaling up by Simplifying GNNs

Other Topics

• Fast Neural Subgraph Matching and Counting
• GNNs for Recommender Systems
• Community Detection in Networks
• Deep Generative Models for Graphs

Reference

• cs224w 강의
• Graph Representation Learning Book