Joint-Multimodal-Embedding
Keras implementation of a Siamese Neural Network for Joint Multimodal Text-Image Embedding
Tested with a Tensorflow backend
This embedding network was used in a larger deep learning project aimed at using GANs to generate video from text descriptions. Check out my blogpost on the results.
Tools Used:
Python 3
Keras
Keras' Xception V1 model pre-trained on ImageNet
Text data vectorized prior to running this network using a Variational Autoencoder and Facebook's fastText pre-trained Word2Vec. More information found here.