Anas1108 / Multimodal_Memes_Classification

Build a PyTorch-based multimodal architecture to classify memes using image & caption. Trained on a meme classification dataset, MLP architecture uses PyTorch, Numpy, Matplotlib, & Sklearn to achieve improved performance compared to baselines.

Multimodal Architecture for Meme Classification

Get the Dataset by clicking:

https://drive.google.com/file/d/1Kr5x8kyaVhFztCB6WM0eBVi0xDxhHSex/view?usp=sharing

Introduction

This project aims to build a multimodal architecture using PyTorch to classify memes as Positive & Very Positive(1), Neutral(0), or Negative & Very Negative(2) based on both image and caption. The architecture is based on a Multilayer Perceptron (MLP) network and will be trained and evaluated on a relevant meme classification dataset.

Dependencies

• PyTorch
• Numpy
• Matplotlib (optional)
• Sklearn (optional)

Dataset

The dataset used for training and evaluating the multimodal architecture will be a meme classification dataset containing images and captions, with multiple examples of memes for each class (Positive & Very Positive, Neutral, or Negative & Very Negative).

Usage

1. Clone the repository git clone https://github.com/Anas1108/Multimodal_Memes_Classification
2. Install the dependencies using pip install -r requirements.txt
3. Open the Jupyter Notebook and run the cells to train the model.
4. Evaluate the model on the test data to see its performance on the meme classification task.

Results

The performance of the multimodal architecture will be evaluated using relevant metrics such as accuracy and compared to baselines to demonstrate its effectiveness in classifying memes based on both image and caption.

Conclusion

This project provides a basic implementation of a multimodal architecture using PyTorch for a meme classification task and serves as a starting point for further exploration and improvement. The use of the MLP network and tools such as Numpy, Matplotlib, and Sklearn allows for improved performance and easy implementation.