satwikkottur

clevr-dialog

Repository to generate CLEVR-Dialog: A diagnostic dataset for Visual Dialog

VisualWord2Vec

Learning visually grounded word embeddings using Abstract scenes

StochasticMCMC

MCMC for posterior distribution sampling

MovieRecommend

A movie recommender system based on Collaborative Filtering and Topic Modeling (LDA)

FluidSimulator

Fluid simulation - Water and Fire interaction

abstract_scenes_v002

The second version of the interface for Abstract Scenes research project.

DeepLearningMovies

Kaggle's competition for using Google's word2vec package for sentiment analysis

DSTC8-AVSD

We rank the 1st in DSTC8 Audio-Visual Scene-Aware Dialog competition. This is the source code for our IEEE/ACM TASLP (AAAI2020-DSTC8-AVSD) paper "Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog".

EgoVLP

[Arxiv2022] Egocentric Video-Language Pretraining

gitignore

A collection of useful .gitignore templates

ImageTextDetector

Fall 2014 Course project for Computer Vision course

lang-emerge-parlai

Implementation of EMNLP 2017 Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using PyTorch and ParlAI

MTN

Code for the paper Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems (ACL19)

mturk-code-samples

Code samples to help you get started with the Amazon Mechanical Turk Requester API

neural-networks-and-deep-learning

Code samples for my book "Neural Networks and Deep Learning"

nn

ParlAI

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.

rnn

Recurrent Neural Network library for Torch7's nn

satwikkottur.github.io

Personal Webpage

simmc

With the aim of building next generation virtual assistants that can handle multimodal inputs and perform multimodal actions, we introduce two new datasets (both in the virtual shopping domain), the annotation schema, the core technical tasks, and the baseline models. The code for the baselines and the datasets will be opensourced.

simmc2

Code for SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

sparse-app

Android App to do sparse reconstruction

tensorflow

Open source software library for numerical computation using data flow graphs.

tutorials

VD-BERT

visdial-1

[CVPR 2017] Torch code for Visual Dialog

visdial-bert

Implementation for "Large-scale Pretraining for Visual Dialog" https://arxiv.org/abs/1912.02379

visdial-challenge-starter-pytorch

Starter code in PyTorch for the Visual Dialog challenge

visual-semantic-embedding

Implementation of the image-sentence embedding method described in "Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models"