ysh329 / bolt

Bolt is a deep learning library with high performance and heterogeneous flexibility.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Introduction

License: MIT

Bolt is a light-weight library for deep learning. Bolt, as a universal deployment tool for all kinds of neural networks, aims to minimize the inference runtime as much as possible. Higher speed, better security and more efficient memory management are the advantages that Bolt strives to provide. Bolt has been widely deployed and used in many departments of HUAWEI company, such as 2012 Laboratory, CBG and HUAWEI Product Lines. Feel free to make good use of issue submission, or join our QQ chatroom (Chinese): 833345709.

Quick Start

Generally, there are two steps to get started with bolt. It's quiet easy for users to quickly running bolt.

  1. Conversion: use X2bolt to convert your model from caffe,onnx,tflite or tensorflow to .bolt;

  2. Inference: run benchmark with .bolt and data to get the inference result.

    For more details about the usage of X2bolt and benchmark tools, see docs/USER_HANDBOOK.md.

Features

  • Support Frameworks

    caffe, onnx, tflite, tensorflow

  • Inference Precision

    Float32, Float16, Int8, 1-bit

  • Hardware

    ARM CPU(v7, v8, v8.2), Mali GPU, X86(AVX2)

  • Verified Networks

    Bolt has shown its high performance in the inference of common CV and NLP neural networks. Some of the representative networks that we have verified are listed below. You can find detailed benchmark information in docs/BENCHMARK.md.

    Application Models
    CV Squeezenet/Mobilenet_v1/Mobilenet_v2/Mobilenet_v3/Resnet50
    /Ghostnet/SSD/Yolov3/Pointnet/...etc.
    NLP Bert/TinyBert/Albert/Neural Machine Translation/Text To Speech
    /Automatic Speech Recognition/...etc.
    More DL Tasks ...

    More models than these mentioned above are supported, users are encouraged to further explore.

  • More Advanced Features

Documentations

Everything you want to know about bolt is recorded in the detailed documentations stored in docs.

Acknowledgement

Bolt refers to the following projects: caffe, onnx, tensorflow, ncnn, mnn, dabnn.

License

The MIT License(MIT)

About

Bolt is a deep learning library with high performance and heterogeneous flexibility.

License:MIT License


Languages

Language:C++ 82.7%Language:C 9.9%Language:Shell 3.7%Language:Python 1.8%Language:CMake 0.8%Language:Java 0.8%Language:Objective-C 0.2%Language:Objective-C++ 0.1%