A Paper List for Recommend-system PreTrained Models

This is a paper list for pretrained recommend System (recommendation) models. It also contains some related research areas.

Keyword: Recommend System, pretrained models

Welcome to open an issue or make a pull request!

Paper List

Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect, arXiv, 2020, [paper]
Self-Supervised Learning for Recommender Systems: A Survey ,arxiv 2022, [paper]
Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems, arxiv 2022, [paper]

Yelp[link]
Petdata[link]
M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining, CVPR 2022

BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer, CIKM 2019 , [paper][code]
S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization , CIKM-2020 , [paper][code]
Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation, Recsys 2021 , [paper][code]
Towards Universal Sequence Representation Learning for Recommender Systems , KDD 2022 , [paper][code]
Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders, WWW 2023, [paper] [code]

Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation, SIGIR 2020 , [paper], [code]
UPRec: User-Aware Pre-training for Recommender Systems ，submitted TKDE in 2021 , [paper]
U-BERT: Pre-training user representations for improved recommendation, AAAI 2021, [paper]
UserBERT: Self-supervised User Representation Learning , arxiv 2021 , [paper]
One4all User Representation for Recommender Systems in E-commerce , arxiv 2021 , [paper]
One Person, One Model, One World: Learning Continual User Representation without Forgetting, SIGIR 2021 , [paper]
Scaling Law for Recommendation Models: Towards General-purpose User Representations , AAAI 2023 , [paper]

Self-supervised Learning for Large-scale Item Recommendations , CIKM 2021 , [paper]
TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedback , arxiv 2022 , [paper]
IntTower: the Next Generation of Two-Tower Model for Pre-Ranking System, CIKM 2022 , [paper][code]

Language Models as Recommender Systems: Evaluations and Limitations , NeurIPS 2021 Workshop ICBINB , [paper]
CTR-BERT: Cost-effective knowledge distillation for billion-parameter teacher models, [paper]
Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) , Recsys 2022 , [paper])
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems ,arxiv 2022 , [paper]
PTab: Using the Pre-trained Language Model for Modeling Tabular Data, arxiv 2022, [paper]

Curriculum Pre-Training Heterogeneous Subgraph Transformer for Top-N Recommendation , arxiv 2021 ,[paper]
Contrastive Pre-Training of GNNs on Heterogeneous Graphs , CIKM 2021 , [paper]
Self-supervised Graph Learning for Recommendation , SIGIR 2021 , [paper]
Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation , AAAI 2021 , [paper]

By Xiangyang Li (xiangyangli@pku.edu.cn) from Peking University.