YuanchenBei / Awesome-Pretraining-for-Graph-Neural-Networks

A curated list of papers on pre-training for graph neural networks (Pre-train4GNN).

Awesome-Pretraining-for-Graph-Neural-Networks

This repository contains a curated list of papers on (or related to) pre-training for graph neural networks (Pre-train4GNN), which are categorized based on their published years, graph types, pre-training strategies, tuning strategies, and applications.

Continuously updating!

---

Papers

Paper Title	Venue	Graph Type	Pre-training Strategy	Tuning Strategy	Application	PDF Link	Code Link
GraphPro: Graph Pre-training and Prompt Learning for Recommendation	WWW 2024	Dynamic	Generative	Prompt	Recommendation	[PDF]	[Code]
Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery	AAAI 2024	Static	MTL	N/A	Biology	[PDF]	[Code]
Unified Pretraining for Recommendation via Task Hypergraphs	WSDM 2024	Hypergraph	Generative	N/A	Recommendation	[PDF]	[Code]
Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training	CIKM 2023	KG	MTL	N/A	Recommendation	[PDF]	N/A
Self-Contrastive Graph Diffusion Network	MM 2023	Static	Contrastive	N/A	General	[PDF]	N/A
All in One: Multi-Task Prompting for Graph Neural Networks	KDD 2023	Static	Generative	Prompt	General	[PDF]	[Code]
Automated 3D Pre-Training for Molecular Property Prediction	KDD 2023	Static	Generative	N/A	Chemistry&Biology	[PDF]	N/A
What’s Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders	KDD 2023	Static	Generative	N/A	General	[PDF]	[Code]
When to Pre-Train Graph Neural Networks? From Data Generation Perspective!	KDD 2023	Static	Generative (Quantification)	N/A	General	[PDF]	[Code]
GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks	WWW 2023	Static	Contrastive	Prompt	General	[PDF]	[Code]
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner	WWW 2023	Static	Generative	N/A	General	[PDF]	[Code]
Structure Pretraining and Prompt Tuning for Knowledge Graph Transfer	WWW 2023	KG	Generative	Prompt	General	[PDF]	[Code]
Protein Representation Learning by Geometric Structure Pretraining	ICLR 2023	Static	Contrastive	N/A	Biology	[PDF]	[Code]
Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting	SIGIR 2023	Text Graph	Contrastive	Prompt	Text Classification	[PDF]	[Code]
Self-Supervised Graph Structure Refinement for Graph Neural Networks	WSDM 2023	Static	Contrastive	GSL	General	[PDF]	[Code]
Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation	RecSys 2023	Static	MTL	N/A	Recommendation	[PDF]	[Code]
MVRACE: Multi-view Graph Contrastive Encoding for Graph Neural Network Pre-training	CogSci 2023	Static	Contrastive	N/A	General	[PDF]	N/A
Curriculum Pre-training Heterogeneous Subgraph Transformer for Top-N Recommendation	TOIS 2023	Heterogeneous	Curriculum	N/A	Recommendation	[PDF]	N/A
GPPT: Graph Pre-training and Prompt Tuning to Generalize Graph Neural Networks	KDD 2022	Static	Generative	Prompt	General	[PDF]	[Code]
GraphMAE: Self-Supervised Masked Graph Autoencoders	KDD 2022	Static	Generative	N/A	General	[PDF]	[Code]
Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries	KDD 2022	KG	Generative	Generative	General	[PDF]	[Code]
KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction	KDD 2022	GT	Generative	N/A	Biology	[PDF]	[Code]
Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting	KDD 2022	ST-Graph	Generative	N/A	Time Series Forecasting	[PDF]	[Code]
Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering	NIPS 2022	Heterogeneous	Clustering	N/A	General	[PDF]	[Code]
Does GNN Pretraining Help Molecular Representation?	NIPS 2022	Molecular	Evaluation	N/A	General	[PDF]	N/A
Pre-training Molecular Graph Representation with 3D Geometry	ICLR 2022	Static	Generative+Contrastive	N/A	Biology	[PDF]	[Code]
Graph Pre-training for AMR Parsing and Generation	ACL 2022	Semantic Graph	Generative	N/A	NLP (AMR)	[PDF]	[Code]
Neural Graph Matching for Pre-training Graph Neural Networks	SDM 2022	Static	Contrastive	N/A	General	[PDF]	[Code]
Pre-train Graph Neural Networks for Brain Network Analysis	Big Data 2022	Static	Contrastive	N/A	Biology	[PDF]	N/A
GCCAD: Graph Contrastive Learning for Anomaly Detection	TKDE 2022	Static	Contrastive	N/A	Anomaly Detection	[PDF]	[Code]
An Adaptive Graph Pre-training Framework for Localized Collaborative Filtering	TOIS 2022	Static	Contrastive	Meta-Learning	Recommendation	[PDF]	N/A
Pre-training Graph Neural Networks for Link Prediction in Biomedical Networks	Bioinformatics 2022	Static	Generative	N/A	Biology	[PDF]	[Code]
Pre-Training of Equivariant Graph Matching Networks with Conformation Flexibility for Drug Binding	Advanced Science 2022	Dynamic	Generative	N/A	Biology	[PDF]	[Code]
Pre-training on Dynamic Graph Neural Networks	Neurocomputing 2022	Dynamic	Generative	N/A	General	[PDF]	[Code]
Adaptive Transfer Learning on Graph Neural Networks	KDD 2021	Static	N/A	Meta Learning	General	[PDF]	[Code]
Pre-training on Large-Scale Heterogeneous Graph	KDD 2021	Heterogeneous	Contrastive	N/A	General	[PDF]	[Code]
Self-supervised Graph-level Representation Learning with Local and Global Structure	ICML 2021	Static	Contrastive	N/A	Biology	[PDF]	[Code]
Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks	IJCAI 2021	Static	Decomposition	N/A	General	[PDF]	N/A
Contrastive Pre-Training of GNNs on Heterogeneous Graphs	CIKM 2021	Heterogeneous	Contrastive	N/A	General	[PDF]	[Code]
Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation	WSDM 2021	Static	Generative+RL	N/A	Recommendation	[PDF]	[Code]
Pre-training Graph Transformer with Multimodal Side Information for Recommendation	MM 2021	Static	Generative	N/A	Recommendation	[PDF]	[Code]
Learning to Pre-train Graph Neural Networks	AAAI 2021	Static	Contrastive	N/A	General	[PDF]	[Code]
GPT-GNN: Generative Pre-Training of Graph Neural Networks	KDD 2020	Static	Generative	N/A	General	[PDF]	[Code]
GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training	KDD 2020	Static	Contrastive	N/A	General	[PDF]	[Code]
Strategies for Pre-training Graph Neural Networks	ICLR 2020	Static	Generative	N/A	Chemistry&Biology	[PDF]	[Code]
Pre-training of Graph Augmented Transformers for Medication Recommendation	IJCAI 2019	Ontology Tree	Generative	N/A	Recommendation	[PDF]	[Code]

---

Pre-print Papers

Paper Title	Venue	Graph Type	Pre-training Strategy	Tuning Strategy	Application	PDF Link	Code Link
GraphGPT: Graph Learning with Generative Pre-trained Transformers	Arxiv 2023	Static	Generative	N/A	General	[PDF]	[Code]
GraphGPT: Graph Instruction Tuning for Large Language Models	Arxiv 2023	Static	N/A	Prompt	General	[PDF]	[Code]
AdapterGNN: Efficient Delta Tuning Improves Generalization Ability in Graph Neural Networks	Arxiv 2023	Static	N/A	Adapter	General	[PDF]	N/A
Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks	Arxiv 2023	Static	N/A	AutoML	General	[PDF]	N/A
GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning	Arxiv 2023	Static	Contrastive	Prompt	General	[PDF]	N/A
CPDG: A Contrastive Pre-Training Method for Dynamic Graph Neural Networks	Arxiv 2023	Dynamic	Contrastive	Enhanced Embedding	General	[PDF]	N/A
Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training	Arxiv 2023	KG	MTL	N/A	Recommendation	[PDF]	N/A
SGL-PT: A Strong Graph Learner with Graph Prompt Tuning	Arxiv 2023	Static	Generative	Prompt	General	[PDF]	N/A
Similarity-aware Positive Instance Sampling for Graph Contrastive Pre-training	Arxiv 2022	Static	Contrastive	N/A	General	[PDF]	N/A
DiP-GNN: Discriminative Pre-Training of Graph Neural Networks	Arxiv 2022	Static	Generative	N/A	General	[PDF]	N/A
Hypergraph Pre-training with Graph Neural Networks	Arxiv 2021	Hypergraph	Contrastive	N/A	General	[PDF]	N/A
GRAPH-BERT: Only Attention is Needed for Learning Graph Representations	Arxiv 2020	Static	Generative	N/A	General	[PDF]	[Code]
Pre-training Graph Neural Networks with Kernels	Arxiv 2018	Static	Kernel	N/A	General	[PDF]	N/A

---

Survey Papers

Paper Title	Venue	PDF Link	Code Link
Graph Prompt Learning: A Comprehensive Survey and Beyond	Arxiv 2023	[PDF]	[Code]
Towards Graph Foundation Models: A Survey and Beyond	Arxiv 2023	[PDF]	N/A
Generative Diffusion Models on Graphs: Methods and Applications	IJCAI 2023	[PDF]	N/A
Graph Meets LLMs: Towards Large Graph Models	Arxiv 2023	[PDF]	[Code]
A Survey of Graph Prompting Methods: Techniques, Applications, and Challenges	Arxiv 2023	[PDF]	N/A
A Survey of Pre-training on Graphs: Taxonomy, Methods and Applications	Arxiv 2022	[PDF]	[Code]
Self-Supervised Learning of Graph Neural Networks: A Unified Review	TPAMI 2022	[PDF]	N/A
Graph Self-Supervised Learning: A Survey	TKDE 2022	[PDF]	N/A
Self-supervised Learning on Graphs: Contrastive, Generative, or Predictive	TKDE 2021	[PDF]	N/A
Self-supervised Learning on Graphs: Deep Insights and New Directions	Arxiv 2020	[PDF]	[Code]

---

Tutorials

Tutorial Title	Venue	PDF Link
Self-supervised Learning and Pre-training on Graphs	WWW 2023	[PDF]