Luolin0826 / band

Multi-DNN Inference Engine for Heterogeneous Mobile Processors

Band: Multi-DNN Framework for Mobile-Cloud Platform

Introduction

Band is an efficient deep learning platform for mobile-cloud collaborative support for multiple DNNs. Band supports backend-agnostic coordination of DNN requests on heterogeneous processors in a mobile device to cloud server. Band is currently backed by following backend machine learning frameworks.

	Tensorflow v2.9.2	...
Android	☑
iOS	☐
gRPC	☐

Band provides Java and C APIs, as well as an official plugin for Unreal Engine.

Useful Links

• Examples (requires update)
• Unreal Engine Plugin

Codebase

.
├── band
│   ├── backend  # backend-specific implementation of `interface`
│   ├── c  # C API
│   ├── docs
│   ├── interface  # Backend-agnostic interfaces. Each backend (e.g., Tensorflow Lite, MNN, ...) should implement them to communicate with Band core
│   ├── java  # Java API
│   ├── scheduler  # Schedulers
│   ├── test  # Test codes
│   ├── tool  # Benchmark tools
├── script  # Utilities
├── third_party
└── WORKSPACE

Getting Started

Prerequisites

• Install Android SDK 28, NDK v19.2.53456

  • or create Visual Studio Code Dev Container using [root]/.devcontainer
  • or utilize [root]/.devcontainer/Dockerfile

• Configure Android SDK, NDK for build system (Bazel)

  python configure.py

How to Build / Run

Refer to detailed instructions in [root]/script

• Run test

  python script/run_test.py -android 

• Build Android AAR

  sh script/build_aar_armv8.sh

• Build C API

  python script/build_c_api.py -android 

• Run benchmark -- check [root]/docs/benchmark.md

Citation

If you find our work useful, please cite our paper below! The original codebase for paper submission is archived here

@inproceedings{jeong2022band,
  title={Band: coordinated multi-DNN inference on heterogeneous mobile processors},
  author={Jeong, Joo Seong and Lee, Jingyu and Kim, Donghyun and Jeon, Changmin and Jeong, Changjin and Lee, Youngki and Chun, Byung-Gon},
  booktitle={Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services},
  pages={235--247},
  year={2022}
}

Acknowledgments

• Tensorflow
• NCNN - CPU affinity control

(back to top)