metaphysicser / PS-Mixer

PS-Mixer: A Polar-Vector and Strength-Vector Mixer Model for Multimodal Sentiment Analysis

PS-Mixer

PS-Mixer: A Polar-Vector and Strength-Vector Mixer Model for Multimodal Sentiment Analysis

We propose a Polar-Vector and Strength-Vector mixer model called PS-Mixer, which is based on MLP-Mixer, to achieve better communication between different modal data for multimodal sentiment analysis.

Requirements

• Python 3.8
• Pytorch 1.11.0

you could run the following command to build the environment.

pip install requirements.txt

Data Download

• Install CMU Multimodal SDK. Ensure, you can perform from mmsdk import mmdatasdk.
• Option 1: Download pre-computed splits and place the contents inside datasets folder.
• Option 2: Re-create splits by downloading data from MMSDK. For this, simply run the code as detailed next.

Running the code

1. Set word_emb_path in config.py to glove file.
2. Set sdk_dir to the path of CMU-MultimodalSDK.
3. python train.py --data mosi. Replace mosi with mosei for other datasets.

Result

Acknowledgements

We begin our work on the basis of MISA initially, so the whole code architecture is similar to it, including the data process, data loader and evaluation metrics. Thanks to their open source spirit for saving us a lot of time.

Contact

For any questions, please email at [e-mail]