Fabrik is an online collaborative platform to build, visualize and train deep learning models via a simple drag-and-drop interface. It allows researchers to collaboratively develop and debug models using a web GUI that supports importing, editing and exporting networks written in widely popular frameworks like Caffe, Keras, and TensorFlow.
This app is presently under active development and we welcome contributions. Please check out our issues thread to find things to work on, or ping us on Gitter.
Installation Instructions
Setting up Fabrik on your local machine is really easy. You can setup Fabrik using two methods:
Using Docker
-
Get the source code on to your machine via git.
git clone https://github.com/Cloud-CV/Fabrik.git && cd Fabrik
-
Rename
settings/dev.sample.py
asdev.py
.cp settings/dev.sample.py settings/dev.py
-
Build and run the Docker containers. This might take a while. You should be able to access Fabrik at http://0.0.0.0:8000.
docker-compose up --build
Setup Authenticaton for Docker Environment
-
Go to Github Developer Applications and create a new application. here
-
For local deployments the following is what should be used in the options:
- Application name: Fabrik
- Homepage URL: http://0.0.0.0:8000
- Application description: Fabrik
- Authorization callback URL: http://0.0.0.0:8000/accounts/github/login/callback/
-
Github will provide you with a Client ID and Secret Key, save these.
-
Create a superuser in django service of docker container
docker-compose run django python manage.py createsuperuser
Note: Before creating make sure that django service of docker image is running, it can be done by executing
docker-compose up
followed byCtrl + C
to save docker configuration. -
Open http://0.0.0.0:8000/admin and login with credentials from step 4.
-
Setting up Social Accounts in django admin
-
Under
Social Accounts
, openSocial applications
and click onAdd Social Application
. -
Choose the
Provider
of social application asGithub
and name itGithub
. -
Add the sites available to the right side, so github is allowed for the current site.
-
Copy and paste your
Client ID
andSecret Key
into the apppropriate fields and Save.
-
-
Go to
Sites
tab and update theDomain name
to0.0.0.0:8000
.
Using Virtual Environment
-
First set up a virtualenv. Fabrik runs on Python2.7.
sudo apt-get install python-pip python-dev python-virtualenv virtualenv --system-site-packages ~/Fabrik --python=python2.7 source ~/Fabrik/bin/activate
-
Clone the repository via git
git clone --recursive https://github.com/Cloud-CV/Fabrik.git && cd Fabrik
-
Rename settings/dev.sample.py as settings/dev.py and change credential in settings/dev.py
cp settings/dev.sample.py settings/dev.py
- Replace the hostname to
localhost
in settings/dev.py line 15. It should now look like this:
'HOST': os.environ.get("POSTGRES_HOST", 'localhost'),
- Replace the hostname to
-
Install redis server
sudo apt-get install redis-server
-
Replace the hostname to
localhost
in settings/common.py line 115."CONFIG": { # replace redis hostname to localhost if running on local system "hosts": [("localhost", 6379)], "prefix": u'fabrik:', },
-
Replace celery result backend in settings/common.py line 122 with localhost.
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
-
Replace celery broker url and result backend hostname to
localhost
in ide/celery_app.py, line 8.app = Celery('app', broker='redis://localhost:6379/0', backend='redis://localhost:6379/0', include=['ide.tasks'])
-
-
If you have Caffe, Keras and Tensorflow already installed on your computer, skip this step
-
For Linux users
-
Install Caffe, Keras and Tensorflow
cd Fabrik/requirements yes Y | sh caffe_tensorflow_keras_install.sh
-
Open your ~/.bashrc file and append this line at the end
export PYTHONPATH=~/caffe/caffe/python:$PYTHONPATH
-
Save, exit and then run
source ~/.bash_profile cd ..
-
-
For Mac users
- Install dependencies
-
For developers:
pip install -r requirements/dev.txt
-
Others:
pip install -r requirements/common.txt
-
Setup postgres database
-
Start postgresql by typing
sudo service postgresql start
-
Now login as user postgres by running
sudo -u postgres psql
and type the commands belowCREATE DATABASE fabrik; CREATE USER admin WITH PASSWORD 'fabrik'; ALTER ROLE admin SET client_encoding TO 'utf8'; ALTER ROLE admin SET default_transaction_isolation TO 'read committed'; ALTER ROLE admin SET timezone TO 'UTC'; ALTER USER admin CREATEDB;
-
Exit psql by typing in \q and hitting enter.
-
-
Migrate
python manage.py makemigrations caffe_app python manage.py migrate
-
Install node modules
npm install npm install --save-dev json-loader sudo npm install -g webpack@1.15.0
- Run the command below in a separate terminal for hot-reloading, ie see the changes made to the UI in real time.
webpack --progress --watch --colors
-
Start celery worker
celery -A ide worker --app=ide.celery_app --loglevel=info
The celery worker needs to be run parallel to the django server in a separate terminal.
-
Start django application
python manage.py runserver
You should now be able to access Fabrik at http://localhost:8000.
Setup Authenticaton for Virtual Environment
-
Go to Github Developer Applications and create a new application. here
-
For local deployments the following is what should be used in the options:
- Application name: Fabrik
- Homepage URL: http://localhost:8000
- Application description: Fabrik
- Authorization callback URL: http://localhost:8000/accounts/github/login/callback/
-
Github will provide you with a client ID and secret, save these.
-
Create a superuser in django
python manage.py createsuperuser
-
Start the application
python manage.py runserver
-
Login with credentials from step 4.
-
Setting up Social Accounts in django admin
-
Under
Social Accounts
openSocial applications
, click onAdd Social Application
. -
Choose the
Provider
of social application asGithub
& name itGithub
. -
Add the sites available to the right side, so github is allowed for the current site. This should be
localhost:8000
for local deployment. -
Copy and paste your
Client ID
andSecret Key
into the apppropriate fields and Save.
-
-
From the django admin home page, go to
Sites
under theSites
category and updateDomain name
tolocalhost:8000
.
Note: For testing, you will only need one authentication backend. However, if you want to try out Google's authentication then, you will need to follow the same steps as above, but switch out the Github for google.
Usage
python manage.py runserver
Example
- Use
example/tensorflow/GoogleNet.pbtxt
for tensorflow import - Use
example/caffe/GoogleNet.prototxt
for caffe import - Use
example/keras/vgg16.json
for keras import
Tested models
The model conversion between currently supported frameworks is tested on some models.
Models | Caffe | Keras | Tensorflow |
---|---|---|---|
Inception V3 | √ | √ | √ |
Inception V4 | √ | √ | √ |
ResNet 101 | √ | √ | √ |
VGG 16 | √ | √ | √ |
GoogLeNet | √ | × | × |
SqueezeNet | √ | × | × |
DenseNet | √ | × | × |
AllCNN | √ | × | × |
AlexNet | √ | √ | √ |
FCN32 Pascal | √ | × | × |
YoloNet | √ | √ | √ |
Pix2Pix | √ | × | × |
VQA | √ | √ | √ |
Denoising Auto-Encoder | × | √ | √ |
Note: For models that use a custom LRN layer (Alexnet), Keras expects the custom layer to be passed when it is loaded from json. LRN.py is located in keras_app/custom_layers. Alexnet import for Keras
Documentation
- Using a Keras model exported from Fabrik
- Loading a Keras model exported from Fabrik and printing its summary
- Using an Exported Caffe Model
- Loading a caffe model in python and printing its parameters and output size
- List of models tested with Fabrik
- Adding model to the Fabrik model zoo
- Adding new layers
- Using custom layers with Keras
- Linux installation walk-through
License
This software is licensed under GNU GPLv3. Please see the included License file. All external libraries, if modified, will be mentioned below explicitly.