UCSB-VRL / GTNet

GTNet:Guided Transformer Network for Detecting Human-Object Interactions

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

GTNet

GTNet: Guided Transformer Network for Detecting Human-Object Interactions

A S M Iftekhar, Satish Kumar, R. Austin McEver, Suya You, B.S. Manjunath.

Paper.

GTNet got accepted to Pattern Recognition and Tracking XXXIV at SPIE commerce+ defence Program.

This codebase only contains code for vcoco dataset.

Our Results on V-COCO dataset

Method mAP (Scenario 1)
VSGNet 51.8
ConsNet 53.2
IDN 53.3
OSGNet 53.4
Sun et al. 55.2
GTNet 58.3

Installation & Setup

  1. Clone repository (recursively):
git clone --recursive https://github.com/UCSB-VRL/GTNet.git
cd GTNet
  1. Please find the data,annotations,object detection results and embeddings for vcoco here. Download it, unzip and setup the path to directory by running:
python3 setup.py -d <full path to the downloaded folder>

Folder description can be found in our old work

  1. Setup enviroment by running (used python 3.6.9):
pip3 install -r requirements.txt
  1. Download the best model from here and keep it inside a folder in the repository. We assume that you put it inside soa_vcoco folder in the repository. You can change it to anything you want.

Inference & Training

All commands need to be run from the scripts folder.

To dump results from the best model:

bash run_inference.sh

Be sure to keep the downloaded best model in soa_vcoco folder in the repository, if you put it some other places, change the bash file accordingly. After that, to get the results in the paper run:

bash run_eval_vcoco.sh

To train with 8 GPUS run:

bash run_train.sh

Please check main.py for various flags.

Please contact Iftekhar (iftekhar@ucsb.edu) for any queries.

About

GTNet:Guided Transformer Network for Detecting Human-Object Interactions


Languages

Language:Python 99.6%Language:Shell 0.4%