MIRCO

PyTorch implementation for paper: Latent Structures Mining with Contrastive Modality Fusion for Multimedia Recommendation

Dependencies

Python 3.6
torch==1.5.0
scikit-learn==0.24.2
torch-scatter==2.0.8

Dataset Preparation

Download 5-core reviews data, meta data, and image features from Amazon product dataset. Put data into the directory data/meta-data/.

Install sentence-transformers and download pretrained models to extract textual features. Unzip pretrained model into the directory sentence-transformers/:

├─ data/: 
    ├── sports/
    	├── meta-data/
    		├── image_features_Sports_and_Outdoors.b
    		├── meta-Sports_and_Outdoors.json.gz
    		├── reviews_Sports_and_Outdoors_5.json.gz
    ├── sentence-transformers/
        	├── stsb-roberta-large

Run python build_data.py to preprocess data.
Run python cold_start.py to build cold-start data.
We provide processed data Baidu Yun (access code: m37q), Google Drive.

Usage

Start training and inference as:

cd codes
python main.py --dataset {DATASET}

For cold-start settings:

python main.py --dataset {DATASET} --core 0 --verbose 1 --lr 1e-5

Citation

If you want to use our codes in your research, please cite:

@article{MICRO21,
  title     = {Latent Structures Mining with Contrastive Modality Fusion for Multimedia Recommendation},
  author    = {Zhang, Jinghao and 
               Zhu, Yanqiao and 
               Liu, Qiang and
               Zhang, Mengqi and
               Wu, Shu and 
               Wang, Liang},
  journal = {arXiv.org},
  year={2021},
  eprint={2111.00678},
  archivePrefix={arXiv},
  primaryClass={cs.IR}
}

Acknowledgement

The structure of this code is largely based on LightGCN. Thank for their work.

weiwei1206 / MICRO