Click-through rate (CTR) prediction is a critical task for many industrial applications such as online advertising, recommender systems, and sponsored search. FuxiCTR provides an open-source library for CTR prediction, with key features in configurability, tunability, and reproducibility. We hope this project could benefit both researchers and practitioners with the goal of open benchmarking for CTR prediction tasks.
-
Configurable: Both data preprocessing and models are modularized and configurable.
-
Tunable: Models can be automatically tuned through easy configurations.
-
Reproducible: All the benchmarks can be easily reproduced.
-
Extensible: It supports both pytorch and tensorflow models, and can be easily extended to any new models.
No |
Publication |
Model |
Paper |
Benchmark |
Version |
|
π Feature Interaction Models |
|
|
|
|
|
1 |
WWW'07 |
LR |
Predicting Clicks: Estimating the Click-Through Rate for New Ads π©Microsoft |
βοΈ |
torch |
2 |
ICDM'10 |
FM |
Factorization Machines |
βοΈ |
torch |
3 |
CIKM'13 |
DSSM |
Learning Deep Structured Semantic Models for Web Search using Clickthrough Data π©Microsoft |
βοΈ |
torch |
4 |
CIKM'15 |
CCPM |
A Convolutional Click Prediction Model |
βοΈ |
torch |
5 |
RecSys'16 |
FFM |
Field-aware Factorization Machines for CTR Prediction π©Criteo |
βοΈ |
torch |
6 |
RecSys'16 |
DNN |
Deep Neural Networks for YouTube Recommendations π©Google |
βοΈ |
torch , tf |
7 |
DLRS'16 |
Wide&Deep |
Wide & Deep Learning for Recommender Systems π©Google |
βοΈ |
torch , tf |
8 |
ICDM'16 |
IPNN |
Product-based Neural Networks for User Response Prediction |
βοΈ |
torch |
9 |
KDD'16 |
DeepCrossing |
Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features π©Microsoft |
βοΈ |
torch |
10 |
NIPS'16 |
HOFM |
Higher-Order Factorization Machines |
βοΈ |
torch |
11 |
IJCAI'17 |
DeepFM |
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction π©Huawei |
βοΈ |
torch , tf |
12 |
SIGIR'17 |
NFM |
Neural Factorization Machines for Sparse Predictive Analytics |
βοΈ |
torch |
13 |
IJCAI'17 |
AFM |
Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks |
βοΈ |
torch |
14 |
ADKDD'17 |
DCN |
Deep & Cross Network for Ad Click Predictions π©Google |
βοΈ |
torch , tf |
15 |
WWW'18 |
FwFM |
Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising π©Oath, TouchPal, LinkedIn, Alibaba |
βοΈ |
torch |
16 |
KDD'18 |
xDeepFM |
xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems π©Microsoft |
βοΈ |
torch |
17 |
CIKM'19 |
FiGNN |
FiGNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction |
βοΈ |
torch |
18 |
CIKM'19 |
AutoInt/AutoInt+ |
AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks |
βοΈ |
torch |
19 |
RecSys'19 |
FiBiNET |
FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction π©Sina Weibo |
βοΈ |
torch |
20 |
WWW'19 |
FGCNN |
Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction π©Huawei |
βοΈ |
torch |
21 |
AAAI'19 |
HFM/HFM+ |
Holographic Factorization Machines for Recommendation |
βοΈ |
torch |
22 |
Arxiv'19 |
DLRM |
Deep Learning Recommendation Model for Personalization and Recommendation Systems π©Facebook |
βοΈ |
torch |
23 |
NeuralNetworks'20 |
ONN |
Operation-aware Neural Networks for User Response Prediction |
βοΈ |
torch , tf |
24 |
AAAI'20 |
AFN/AFN+ |
Adaptive Factorization Network: Learning Adaptive-Order Feature Interactions |
βοΈ |
torch |
25 |
AAAI'20 |
LorentzFM |
Learning Feature Interactions with Lorentzian Factorization π©eBay |
βοΈ |
torch |
26 |
WSDM'20 |
InterHAt |
Interpretable Click-through Rate Prediction through Hierarchical Attention π©NEC Labs, Google |
βοΈ |
torch |
27 |
DLP-KDD'20 |
FLEN |
FLEN: Leveraging Field for Scalable CTR Prediction π©Tencent |
βοΈ |
torch |
28 |
CIKM'20 |
DeepIM |
Deep Interaction Machine: A Simple but Effective Model for High-order Feature Interactions π©Alibaba, RealAI |
βοΈ |
torch |
29 |
WWW'21 |
FmFM |
FM^2: Field-matrixed Factorization Machines for Recommender Systems π©Yahoo |
βοΈ |
torch |
30 |
WWW'21 |
DCN-V2 |
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems π©Google |
βοΈ |
torch |
31 |
CIKM'21 |
DESTINE |
Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction π©Alibaba |
βοΈ |
torch |
32 |
CIKM'21 |
EDCN |
Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models π©Huawei |
βοΈ |
torch |
33 |
DLP-KDD'21 |
MaskNet |
MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask π©Sina Weibo |
βοΈ |
torch |
34 |
SIGIR'21 |
SAM |
Looking at CTR Prediction Again: Is Attention All You Need? π©BOSS Zhipin |
βοΈ |
torch |
35 |
KDD'21 |
AOANet |
Architecture and Operation Adaptive Network for Online Recommendations π©Didi Chuxing |
βοΈ |
torch |
36 |
AAAI'23 |
FinalMLP |
FinalMLP: An Enhanced Two-Stream MLP Model for CTR Prediction :triangular_flag_on_post:Huawei |
βοΈ |
torch |
37 |
SIGIR'23 |
FINAL |
FINAL: Factorized Interaction Layer for CTR Prediction :triangular_flag_on_post:Huawei |
βοΈ |
torch |
|
π Behavior Sequence Modeling |
|
|
|
|
|
38 |
KDD'18 |
DIN |
Deep Interest Network for Click-Through Rate Prediction π©Alibaba |
βοΈ |
torch |
39 |
AAAI'19 |
DIEN |
Deep Interest Evolution Network for Click-Through Rate Prediction :triangular_flag_on_post:Alibaba |
βοΈ |
torch |
40 |
DLP-KDD'19 |
BST |
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba :triangular_flag_on_post:Alibaba |
βοΈ |
torch |
41 |
CIKM'20 |
DMIN |
Deep Multi-Interest Network for Click-through Rate Prediction :triangular_flag_on_post:Alibaba |
|
torch |
42 |
AAAI'20 |
DMR |
Deep Match to Rank Model for Personalized Click-Through Rate Prediction :triangular_flag_on_post:Alibaba |
|
torch |
43 |
Arxiv'21 |
ETA |
End-to-End User Behavior Retrieval in Click-Through RatePrediction Model :triangular_flag_on_post:Alibaba |
|
torch |
44 |
CIKM'22 |
SDIM |
Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction :triangular_flag_on_post:Meituan |
|
torch |
|
π Dynamic Weight Network |
|
|
|
|
|
45 |
NeurIPS'22 |
APG |
APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction π©Alibaba |
|
torch |
46 |
Arxiv'23 |
PPNet |
PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information π©KuaiShou |
|
torch |
|
π Multi-Task Modeling |
|
|
|
|
|
47 |
MachineLearn'97 |
SharedBottom |
Multitask Learning |
|
torch |
48 |
KDD'18 |
MMoE |
Modeling Task Relationships in Multi-task Learning with Multi-Gate Mixture-of-Experts π©Google |
|
torch |
49 |
KDD'18 |
PLE |
Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations π©Tencent |
|
torch |
|
π Multi-Domain Modeling |
|
|
|
|
|
50 |
Arxiv'23 |
PEPNet |
PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information π©KuaiShou |
|
torch |
FuxiCTR has the following dependency requirements.
- python 3.6+
- pytorch 1.0/1.10+ (required only for torch models)
- tensorflow 2.1+ (required only for tf models)
Other packages can be installed via pip install -r requirements.txt
.
-
Run the demo examples
Examples are provided in the demo directory to show some basic usage of FuxiCTR. Users can run the examples for quick start and to understand the workflow.
cd demo
python example1_build_dataset_to_h5.py
python example2_DeepFM_with_h5_input.py
-
Run an existing model
Users can easily run each model in the model zoo following the commands below, which is a demo for running DCN. In addition, users can modify the dataset config and model config files to run on their own datasets or with new hyper-parameters. More details can be found in the readme file.
cd model_zoo/DCN/DCN_torch
python run_expid.py --expid DCN_test --gpu 0
# Change `MODEL` according to the target model name
cd model_zoo/MODEL_PATH
python run_expid.py --expid MODEL_test --gpu 0
-
Implement a new model
The FuxiCTR code structure is modularized, so that every part can be overwritten by users according to their needs. In many cases, only the model class needs to be implemented for a new customized model. If data preprocessing or data loader is not directly applicable, one can also overwrite a new one through the core APIs. We show a concrete example which implements our new model FinalMLP that has been recently published in AAAI 2023. More examples can be found in the model zoo.
π If you find our code or benchmarks helpful in your research, please kindly cite the following papers.
Jieming Zhu, Jinyang Liu, Shuai Yang, Qi Zhang, Xiuqiang He. Open Benchmarking for Click-Through Rate Prediction. The 30th ACM International Conference on Information and Knowledge Management (CIKM), 2021. [Bibtex]
Jieming Zhu, Quanyu Dai, Liangcai Su, Rong Ma, Jinyang Liu, Guohao Cai, Xi Xiao, Rui Zhang. BARS: Towards Open Benchmarking for Recommender Systems. The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2022. [Bibtex]
Welcome to join our WeChat group for any question and discussion. We also have open positions for internships and full-time jobs. If you are interested in research and practice in recommender systems, please reach out via our WeChat group.
![Scan QR code](https://camo.githubusercontent.com/3d295add80442b2593843aca72e0d91ba321ccda34240b54761ed8605c1b51f2/68747470733a2f2f6f70656e62656e63686d61726b2e6769746875622e696f2f424152532f5f696d616765732f7765636861742e6a7067)