Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval
PyTorch implementation of our D2S model for zero-shot sketch-based image retrieval:
Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval
Sounak Dey, Pau Riba, Anjan Dutta, Josep Llados and Yi-Zhe Song
CVPR, 2019
Retrieval Results
Sketchy
TU-Berlin
QuickDraw
Prerequisites
- Linux (tested on Ubuntu 16.04)
- NVIDIA GPU + CUDA CuDNN
- 7z
sudo apt-get install p7zip-fullGetting Started
Introduction
We took the first steps to move towards practical zero shot sketch based image retrieval systems (see the paper for more detail). To this end, we have used Quick Draw! to curate the sketches and for the images we would like to thanks Flickr API for such an amazing API.
The structure of this repo is as follows:
- Installation
- Getting the data
- How to train models
- At last how to test and evaluate
Installation
- Clone this repository
git clone https://github.com/sounakdey/doodle2search.git
cd doodle2search- Install the requirements (not checked)
pip3 install -r requirements.txtDownload datasets
- Sketchy
- TU-Berlin
bash download_datasets.shTrain
Finally we are ready to train. Magical words are:
python3 src/train.py sketchy_extended --data_path <mention the data path of the dataset>The first argument is the dataset name, which you can replace it with tuberlin_extend or quickdraw_extend.
You can check the options.py for changing a lot of the options such dimension size, different models, hyperparameters, etc.
Test
Sketchy
python3 src/test.py sketchy_extend --data_path <mention the data path of the dataset> --load <path of the trained models>Citation
@InProceedings{Dey_2019_CVPR,
author = {Dey, Sounak and Riba, Pau and Dutta, Anjan and Llados, Josep and Song, Yi-Zhe},
title = {Doodle to Search: Practical Zero-Shot Sketch-Based Image Retrieval},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}
Conclusion
Thank you and sorry for the bugs!