monologg / hashtag-prediction-pytorch

Multimodal Hashtag Prediction with instagram data & pytorch (2nd Place on OpenResource Hackathon 2019)

Hashtag Prediction with Pytorch

Multimodal hashtag prediction from instagram

Overview

Model Architecture

• ALBERT for text, VGG16 for image
• Take out the [CLS] token from ALBERT, and change it to 100 dim
• Change to 100 dim after flattening the VGG16 output
• Concat them, and predict among 100 labels.

Dataset

• Collect 50000 data from instagram (w/ selenium crawler)
• Only include English data

How to use

1. Run docker server

$ docker run -d -p 80:80 adieujw/hashtag:latest

2. Put image at Google Drive

• Put your image in this google drive

• Copy the id. It will be used when you give request.

  

3. Request

# URL
localhost:80/predict?image_id=1DGu9R5a9jpkY-fy79VrGFmCdJigzTMC-&text=20%20days%20till%20Christmas%20%F0%9F%98%8D%F0%9F%8E%85&max_seq_len=20&n_label=10

Run on Ainize

1. image_id : the share id you can get from google drive above
2. text : like caption in instagram
3. max_seq_len: maximum sequence length
4. n_label: num of labels you want to predict

https://endpoint.ainize.ai/monologg/hashtag/predict?image_id={image_id}&text={text}&max_seq_len={max_seq_len}&n_label={n_label}

# URL
https://endpoint.ainize.ai/monologg/hashtag/predict?image_id=1DGu9R5a9jpkY-fy79VrGFmCdJigzTMC-&text=20%20days%20till%20Christmas%20%F0%9F%98%8D%F0%9F%8E%85&max_seq_len=20&n_label=10

Result on html

Reference

• Huggingface Transformers
• ALBERT Paper