roza / inversecooking

Recipe Generation from Food Images

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Inverse Cooking: Recipe Generation from Food Images

Code supporting the paper:

Amaia Salvador, Michal Drozdzal, Xavier Giro-i-Nieto, Adriana Romero. Inverse Cooking: Recipe Generation from Food Images. CVPR 2019

If you find this code useful in your research, please consider citing using the following BibTeX entry:

author = {Salvador, Amaia and Drozdzal, Michal and Giro-i-Nieto, Xavier and Romero, Adriana},
title = {Inverse Cooking: Recipe Generation From Food Images},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}


This code uses Python 3.6 and PyTorch 0.4. cuda version 9.0.

  • Installing PyTorch:
$ conda install pytorch torchvision cuda90 -c pytorch
  • Install dependencies
$ pip install -r requirements.txt

Pretrained model

  • Download ingredient and instruction vocabularies here and here, respectively.
  • Download pretrained model here.


You can use our pretrained model to get recipes for your images.

Download the required files (listed above), place them under the data directory, and try our demo notebook src/demo.ipynb.

Note: The demo will run on GPU if a device is found, else it will use CPU.


  • Download Recipe1M (registration required)
  • Extract files somewhere (we refer to this path as path_to_dataset).
  • The contents of path_to_dataset should be the following:

Note: all python calls below must be run from ./src

Build vocabularies

$ python --recipe1m_path path_to_dataset

Images to LMDB (Optional, but recommended)

For fast loading during training:

$ python utils/ --recipe1m_path path_to_dataset

If you decide not to create this file, use the flag --load_jpeg when training the model.


Create a directory to store checkpoints for all models you train (e.g. ../checkpoints and point --save_dir to it.)

We train our model in two stages:

  1. Ingredient prediction from images
python --model_name im2ingr --batch_size 150 --finetune_after 0 --ingrs_only \
--es_metric iou_sample --perminv_ingrs --loss_weight 0 1000.0 1.0 1.0 \
--learning_rate 1e-4 --scale_learning_rate_cnn 1.0 \
--save_dir ../checkpoints --recipe1m_dir path_to_dataset
  1. Recipe generation from images and ingredients (loading from 1.)
python --model_name model --batch_size 256 --recipe_only --transfer_from im2ingr \
--save_dir ../checkpoints --recipe1m_dir path_to_dataset

Check training progress with Tensorboard from ../checkpoints:

$ tensorboard --logdir='../tb_logs' --port=6006


  • Save generated recipes to disk with python --model_name model --greedy --eval_split test.
  • This script will return ingredient metrics (F1 and IoU)


inversecooking is released under MIT license, see LICENSE for details.

ezoic increase your site revenue


Recipe Generation from Food Images

License:MIT License


Language:Python 93.7%Language:Jupyter Notebook 6.3%