Sinp17 / Awesome-Model-Compression-in-Recommender-Systems

A collection of high-quality(newest) papers of model compression in the field of recommender systems

Awesome-Model-Compression-in-Recommender-Systems

A collection of high-quality(newest) papers of model compression, particularly in the field of recommender systems. Due to the discreteness of input data and the characteristics of massive users and items in recommendation scenarios, the compression technics or implementations are quite different from those commonly used in Computer Vision tasks. And this is the why this repro is created.

This repo aims to provide the info for model compression research (especially in RS). Welcome to PR the works (papers, repositories) that are missed by the repo.

Feature

• Focus on compression techniques in Recommender Systems.
• We propose two criteria to classify these papers.

Divide by compression techniques

Parameter Pruning

1. DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving[WSDM 2021]
2. UMEC: UNIFIED MODEL AND EMBEDDING COMPRES- SION FOR EFFICIENT RECOMMENDATION SYSTEMS[ICLR 2021]
3. LEARNABLE EMBEDDING SIZES FOR RECOMMENDER SYSTEMS[ICLR 2021]
4. Single-Shot Embedding Dimension Search in Recommender System[SIGIR 2022]
5. OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction[CIKM 2022]

Scalar Quantization/Binarization

1. Learning Multi-Granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems[WWW 2020]
2. Learning Binarized Graph Representations with Multi-Faceted Quantization Reinforcement for Top-K Recommendation[KDD 2022]
3. Towards Low-Loss 1-Bit Quantization of User-Item Representations for Top-K Recommendation[ArXiv]
4. Post-Training 4-bit Quantization on Embedding Tables[NIPS 2019]

Product quantization

1. Product Quantization for Nearest Neighbor Search
2. Product Quantized Collaborative Filtering[TKDE 2020]
3. LightRec: A Memory and Search-Efficient Recommender System[WWW 2020]
4. Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems[KDD 2021]
5. XLightFM: Extremely Memory-Efficient Factorization Machine[SIGIR 2021]
6. Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings[SIGIR 2022]
7. Learning Elastic Embeddings for Customizing On-Device Recommenders[KDD 2021]
8. Lightweight Self-Attentive Sequential Recommendation[CIKM 2021]

Embedding Factorization

1. Fast and Memory-Efficient Tucker Decomposition for Answering Diverse Time Range Queries[KDD 2021]
2. TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models[MlSys 2021]
3. Next Point-of-Interest Recommendation on Resource-Constrained Mobile Devices[WWW 2020]
4. On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation[SIGIR 2022]

Distillation(could be used for Compression)

1. Rocket Launching: A Universal and Efficient Framework for Training Well-Performing Light Net[AAAI 2018]
2. Ensembled CTR Prediction via Knowledge Distillation[CIKM 2020]
3. Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System[KDD 2018]
4. Collaborative Distillation for Top-N Recommendation[ICDM 2019]
5. DE-RRD: A Knowledge Distillation Framework for Recommender System[CIKM 2020]
6. Bidirectional Distillation for Top-K Recommender System[WWW 2021]
7. ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-Based Recommendation[RecSys 2020]
8. Topology Distillation for Recommender System[KDD 2021]
9. Interpolative Distillation for Unifying Biased and Debiased Recommendation[SIGIR 2022]
10. Unbiased Knowledge Distillation for Recommendation[WSDM 2023]

others

1. Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems[ArXiv 2019]
2. Saec: Similarity-Aware Embedding Compression in Recommendation Systems[ArXiv 2019]
3. A Generic Network Compression Framework for Sequential Recommender Systems[SIGIR 2020]
4. NewsBERT: Distilling Pre-Trained Language Model for Intelligent News Application[EMNLP 2021]

Divide by specific task

general recommendation/CTR-prediction

1. DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving[WSDM 2021]
2. UMEC: UNIFIED MODEL AND EMBEDDING COMPRES- SION FOR EFFICIENT RECOMMENDATION SYSTEMS[ICLR 2021]
3. Single-Shot Embedding Dimension Search in Recommender System[SIGIR 2022]
4. OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction[CIKM 2022
5. Learning Multi-Granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems[WWW 2020]
6. Product Quantized Collaborative Filtering[TKDE 2020]
7. LightRec: A Memory and Search-Efficient Recommender System[WWW 2020]
8. XLightFM: Extremely Memory-Efficient Factorization Machine[SIGIR 2021]
9. Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings[SIGIR 2022]
10. Rocket Launching: A Universal and Efficient Framework for Training Well-Performing Light Net[AAAI 2018]
11. Ensembled CTR Prediction via Knowledge Distillation[CIKM 2020]

sequential recommendation/session-based RS

1. A Generic Network Compression Framework for Sequential Recommender Systems[SIGIR 2020]
2. Next Point-of-Interest Recommendation on Resource-Constrained Mobile Devices[WWW 2020]
3. On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation[SIGIR 2022]
4. Lightweight Self-Attentive Sequential Recommendation[CIKM 2021]

---