msalvaris / DaskStyleTransfer

Running Style Transfer on Dask with PyTorch

Using Dask for Style Transfer

In this repository is a demo on how to use Dask to apply Style Transfer in PyTorch.

All needed commands are in the Makefile

Requirements

Ubuntu PC/VM
Docker
Nvidia runtime for Docker
One or more GPUs

Getting Started

Before you do anything you will need to modify the makefile.

• First edit data_volume and replace /mnt/pipelines with a location on your computer where you will read the data from and write the data to. This will be mapped to /data inside the container.
• Next edit filepath. This is the location as it appears inside the docker container. As it is set by default inside the makefile the location is /data/people. The location people contains a number of files which will be processed by the model. /data/people will actually match to /mnt/pipelines/people outside the container.
• Edit output_path. This should be where there results will be written to.
• Place files you want to run Style Transfer against in the folder you are mapping from in filepath. This is by default /mnt/pipelines/people

Then you must build the container in which we will execute everything.

make build

Then run the container

make run

Then we start the Dask scheduler

make start-scheduler

This also creates a tmux session named dask

Then we start the Dask workers

make start-workers

Each Dask worker will bind to a specific GPU

Finally we run the pipeline:

make run-pipeline

You should be able to view the Dask dashboard if you point your browser to port 8787 of your VM/PC.

You can then stop everything by simply running

make stop

Extra Commands

You can run bash in the container by running

make bash

Notes

By default this demo is set up to use 4 GPUs, make sure you edit the makefile and adjust to your setup