cmu-catalyst / collage

System for automated integration of deep learning backends.

Collage

System for automated integration of deep learning backends.

Installation

Since our implementation uses TVM as the main code generator, install tvm under tvm/.

1. Install dependencies

sudo apt-get update
sudo apt-get install -y python3 python3-dev python3-setuptools gcc libtinfo-dev zlib1g-dev build-essential cmake libedit-dev libxml2-dev

pip3 install --user numpy decorator attrs tornado psutil xgboost cloudpickle pytest

2. Install backends (e.g., cuDNN, MKL,...).

• For cuDNN, install front-end library (commit 360d6e7164dfb7c802493fd1c0464f0d815b852a, tag: v0.1)

3. Create build directory and go to build directory

mkdir tvm/build && cd tvm/build

4. Prepare cmake configuration file. Make sure backend libaries of interest are built together. We provide cmake config that we used for our GPU/CPU experiments (config.cmake.gpu, config.cmake.cpu) in tvm/cmake/. Users may copy it to their build directory and rename it to config.cmake

cp ../cmake/config.cmake.gpu config.cmake

5. Run cmake and make

cmake .. && make -j$(nproc)

6. Declare following environment variables

export COLLAGE_HOME=/path/to/collage/repo
export COLLAGE_TVM_HOME=${COLLAGE_HOME}/tvm
export PYTHONPATH=${COLLAGE_TVM_HOME}/python:${COLLAGE_HOME}/python:${PYTHONPATH}

For more details, please reference TVM installation guide

7. Install Collage dependencies

pip3 install --user graphviz bitarray deap pandas

If you face issues, try

pip3 install --upgrade protobuf
sudo apt-get install graphviz

Demo

Install the following dependencies for deep learning models used for demo.

pip3 install --user torch==1.7.1 torchvision==0.8.2 tqdm protobuf onnx onnxruntime opencv-python transformers

We provide two demos (demo_performance.py, demo_customization.py) under demo/.

• demo_performance.py shows how collage optimizes given workloads with popular backends that Collage provides by default.
• demo_customization.py shows how users can register new backend with their custom codegen, pattern, pattern rule.

For the best result, it is highly recommend to create the tuning log by using autotune_tvm_ops.py before running those demos.

Note

• As Collage uses TVM as its code generator, it cannot support backends that TVM is unable to build. Tested backends are
  • TVM w/o tuning
  • TVM w/ AutoTVM
  • cuDNN v8.0.5
  • cuBLAS v11.3.0
  • TensorRT v7.2.2
  • MKL
  • DNNL
• Since Collage is essentially a profile-guided search, variance in the measurement may affect the final backend placement. For the best result, multiple runs are highly recommended.
• Due to some issues in the implementation, current evolutionary search only supports network implemented in get_network_from_torch(). If an user want to try new network, the network must be implemented within this function.

Cite

@article{jeon2021collage,
  title={Collage: Automated Integration of Deep Learning Backends},
  author={Jeon, Byungsoo and Park, Sunghyun and Liao, Peiyuan and Xu, Sheng and Chen, Tianqi and Jia, Zhihao},
  journal={arXiv preprint arXiv:2111.00655},
  year={2021}
}