luckily-lzy / lite.ai

Lite.AI πŸš€πŸš€πŸŒŸ is a user friendly C++ lib of awesome AI models. such as YOLOXπŸ”₯, YoloRπŸ”₯, YOLOP, YoloV5, YoloV4, DeepLabV3, ArcFace, CosFace, RetinaFace, SSD, etc.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Lite.AI πŸš€πŸš€πŸŒŸ


Introduction.

Lite.AI πŸš€πŸš€πŸŒŸ is a simple and user-friendly C++ library of awesomeπŸ”₯πŸ”₯πŸ”₯ AI models. It's a collection of personal interests. such as YOLOX, YoloV5, YoloV4, DeepLabV3, ArcFace, etc. Lite.AI based on onnxruntime c++ by default. I do have plans to reimplement it with ncnn and MNN, but not coming soon. It includes object detection, face detection, style transfer, face alignment, face recognition, segmentation, colorization, face attributes analysis, image classification, matting, etc. You can use these awesome models simply through lite::cv::Type::Class syntax, such as lite::cv::detection::YoloV5.

Citations.

Cite it as follows if you use Lite.AI. Star πŸŒŸπŸ‘†πŸ» this repo if it does any helps to you ~ πŸ™ƒπŸ€ͺπŸ€

@misc{lite.ai2021,
  title={Lite.AI: A simple and user friendly C++ library of awesome AI models.},
  url={https://github.com/DefTruth/lite.ai},
  note={Open-source software available at https://github.com/DefTruth/lite.ai},
  author={Yan Jun},
  year={2021}
}

Important Notes !!!

Expand for More Notes.

More Notes !!!

  • βœ… (20210815) Added EfficientDet for object detection! See demo.
  • βœ… (20210808) Added ScaledYoloV4 for object detection! See demo.
  • βœ… (20210807) Added TinyYoloV4VOC for object detection! See demo.
  • βœ… (20210807) Added TinyYoloV4COCO for object detection! See demo.
  • βœ… (20210722) Update lite.ai.hub.onnx.md ! Lite.AI contains 70+ AI models with 150+ .onnx files now.
  • ⚠️ (20210802) Added GPU Compatibility for CUDAExecutionProvider. See issue#10.
  • ⚠️ (20210801) fixed issue#9 YOLOX inference error for non-square shape. See yolox.cpp.
  • βœ… (20210801) Added FaceBoxes for face detection! See demo.
  • βœ… (20210727) Added MobileNetV2SE68、PFLD68 for 68 facial landmarks detection! See demo.
  • βœ… (20210726) Added PFLD98 for 98 facial landmarks detection! See demo.
  • ⚠️ (20210716) Lite.AI was rename from the LiteHub repo! LiteHub will no longer be maintained.

Working Notes. πŸ‘‡πŸ»

1. Build Lite.AI.

Build the shared lib of Lite.AI for MacOS from sources. Note that Lite.AI uses onnxruntime as default backend, for the reason that onnxruntime supports the most of onnx's operators.

Linux and Windows.

Linux and Windows.

⚠️ Lite.AI is not directly support Linux and Windows now. For Linux and Windows, you need to build or download(if have official builts) the shared libs of OpenCV and ONNXRuntime firstly and put then into the third_party directory. Please reference the build-docs1 for third_party.

  • Windows: You can reference to issue#6
  • Linux: The Docs and Docker image for Linux will be coming soon ~ issue#2
  • Happy News !!! : πŸš€ You can download the latest ONNXRuntime official built libs of Windows, Linux, MacOS and Arm !!! Both CPU and GPU versions are available. No more attentions needed pay to build it from source. Download the official built libs from v1.8.1. I have used version 1.7.0 for Lite.AI now, you can downlod it from v1.7.0, but version 1.8.1 should also work, I guess ~ πŸ™ƒπŸ€ͺπŸ€. For OpenCV, try to build from source(Linux) or down load the official built(Windows) from OpenCV 4.5.3. Then put the includes and libs into third_party directory of Lite.AI.
  • Clone the Lite.AI from sources:
git clone --depth=1 https://github.com/DefTruth/lite.ai.git  # latest
  • Build shared lib.
cd lite.ai
sh ./build.sh
Expand for more details of How to link the shared lib of Lite.AI?
cd ./build/lite.ai/lib && otool -L liblite.ai.0.0.1.dylib 
liblite.ai.0.0.1.dylib:
        @rpath/liblite.ai.0.0.1.dylib (compatibility version 0.0.1, current version 0.0.1)
        @rpath/libopencv_highgui.4.5.dylib (compatibility version 4.5.0, current version 4.5.2)
        @rpath/libonnxruntime.1.7.0.dylib (compatibility version 0.0.0, current version 1.7.0)
        ...
cd ../ && tree .
β”œβ”€β”€ bin
β”œβ”€β”€ include
β”‚   β”œβ”€β”€ lite
β”‚   β”‚   β”œβ”€β”€ backend.h
β”‚   β”‚   β”œβ”€β”€ config.h
β”‚   β”‚   └── lite.h
β”‚   └── ort
└── lib
    └── liblite.ai.0.0.1.dylib
  • Run the built examples:
cd ./build/lite.ai/bin && ls -lh | grep lite
-rwxr-xr-x  1 root  staff   301K Jun 26 23:10 liblite.ai.0.0.1.dylib
...
-rwxr-xr-x  1 root  staff   196K Jun 26 23:10 lite_yolov4
-rwxr-xr-x  1 root  staff   196K Jun 26 23:10 lite_yolov5
...
./lite_yolov5
LITEORT_DEBUG LogId: ../../../hub/onnx/cv/yolov5s.onnx
=============== Input-Dims ==============
...
detected num_anchors: 25200
generate_bboxes num: 66
Default Version Detected Boxes Num: 5
  • To link lite.ai shared lib. You need to make sure that OpenCV and onnxruntime are linked correctly. Just like:
cmake_minimum_required(VERSION 3.17)
project(testlite.ai)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_BUILD_TYPE debug)
# link opencv.
set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/opencv/lib/cmake/opencv4)
find_package(OpenCV 4 REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
# link onnxruntime.
set(ONNXRUNTIME_DIR ${CMAKE_SOURCE_DIR}/onnxruntime/)
set(ONNXRUNTIME_INCLUDE_DIR ${ONNXRUNTIME_DIR}/include)
set(ONNXRUNTIME_LIBRARY_DIR ${ONNXRUNTIME_DIR}/lib)
include_directories(${ONNXRUNTIME_INCLUDE_DIR})
link_directories(${ONNXRUNTIME_LIBRARY_DIR})
# link lite.ai.
set(LITEHUB_DIR ${CMAKE_SOURCE_DIR}/lite.ai)
set(LITEHUB_INCLUDE_DIR ${LITEHUB_DIR}/include)
set(LITEHUB_LIBRARY_DIR ${LITEHUB_DIR}/lib)
include_directories(${LITEHUB_INCLUDE_DIR})
link_directories(${LITEHUB_LIBRARY_DIR})
# add your executable
add_executable(lite_yolov5 test_lite_yolov5.cpp)
target_link_libraries(lite_yolov5 lite.ai onnxruntime ${OpenCV_LIBS})

A minimum example to show you how to link the shared lib of Lite.AI correctly for your own project can be found at lite.ai-release .

2. Model Zoo.

Lite.AI contains 70+ AI models with 150+ frozen pretrained .onnx files now. Note that the models here are all from third-party projects. Most of the models were converted by Lite.AI. In Lite.AI, different names of the same algorithm mean that the corresponding models come from different repositories, different implementations, or use different training data, etc. βœ… means passed the test and ⚠️ means not implements yet but coming soon. For classes which denoted βœ…, you can use it through lite::cv::Type::Class syntax, such as lite::cv::detection::YoloV5 . More details can be found at Examples for Lite.AI .

Expand Details for Namespace and Lite.AI modules.

Namespace and Lite.AI modules.

Lite.AI contains 70+ AI models with 150+ frozen pretrained .onnx files now. They come from different fields of computer vision. Click the Expand ▢️ button for more details.

Namepace Details
lite::cv::detection Object Detection. one-stage and anchor-free detectors, YoloV5, YoloV4, SSD, etc. βœ…
lite::cv::classification Image Classification. DensNet, ShuffleNet, ResNet, IBNNet, GhostNet, etc. βœ…
lite::cv::faceid Face Recognition. ArcFace, CosFace, CurricularFace, etc. ❇️
lite::cv::face Face Analysis. detect, align, pose, attr, etc. ❇️
lite::cv::face::detect Face Detection. UltraFace, RetinaFace, FaceBoxes, PyramidBox, etc. ❇️
lite::cv::face::align Face Alignment. PFLD(106), FaceLandmark1000(1000 landmarks), PRNet, etc. ❇️
lite::cv::face::pose Head Pose Estimation. FSANet, etc. ❇️
lite::cv::face::attr Face Attributes. Emotion, Age, Gender. EmotionFerPlus, VGG16Age, etc. ❇️
lite::cv::segmentation Object Segmentation. Such as FCN, DeepLabV3, etc. ⚠️
lite::cv::style Style Transfer. Contains neural style transfer now, such as FastStyleTransfer. ⚠️
lite::cv::matting Image Matting. Object and Human matting. ⚠️
lite::cv::colorization Colorization. Make Gray image become RGB. ⚠️
lite::cv::resolution Super Resolution. ⚠️

Lite.AI's Classes and Pretrained Files.

Correspondence between the classes in Lite.AI and pretrained model files can be found at lite.ai.hub.onnx.md. For examples, the pretrained model files for lite::cv::detection::YoloV5 and lite::cv::detection::YoloX are listed as follows.

Class Pretrained ONNX Files Rename or Converted From (Repo) Size
lite::cv::detection::YoloV5 yolov5l.onnx yolov5 (πŸ”₯πŸ”₯πŸ’₯↑) 188Mb
lite::cv::detection::YoloV5 yolov5m.onnx yolov5 (πŸ”₯πŸ”₯πŸ’₯↑) 85Mb
lite::cv::detection::YoloV5 yolov5s.onnx yolov5 (πŸ”₯πŸ”₯πŸ’₯↑) 29Mb
lite::cv::detection::YoloV5 yolov5x.onnx yolov5 (πŸ”₯πŸ”₯πŸ’₯↑) 351Mb
lite::cv::detection::YoloX yolox_x.onnx YOLOX (πŸ”₯πŸ”₯!!↑) 378Mb
lite::cv::detection::YoloX yolox_l.onnx YOLOX (πŸ”₯πŸ”₯!!↑) 207Mb
lite::cv::detection::YoloX yolox_m.onnx YOLOX (πŸ”₯πŸ”₯!!↑) 97Mb
lite::cv::detection::YoloX yolox_s.onnx YOLOX (πŸ”₯πŸ”₯!!↑) 34Mb
lite::cv::detection::YoloX yolox_tiny.onnx YOLOX (πŸ”₯πŸ”₯!!↑) 19Mb
lite::cv::detection::YoloX yolox_nano.onnx YOLOX (πŸ”₯πŸ”₯!!↑) 3.5Mb

It means that you can load the the any one yolov5*.onnx and yolox_*.onnx according to your application through the same Lite.AI classes, such as YoloV5, YoloX, etc.

auto *yolov5 = new lite::cv::detection::YoloV5("yolov5x.onnx");  // for server
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5l.onnx"); 
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5m.onnx");  
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5s.onnx");  // for mobile device 
auto *yolox = new lite::cv::detection::YoloX("yolox_x.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_l.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_m.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_s.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_tiny.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_nano.onnx");  // 3.5Mb only !
  • Downloads:
    Baidu Drive code: 8gin && Google Drive .
    Note, I can not upload all the *.onnx files because of the storage limitation of Google Driver (15G).
  • Object Detection.
Class Size From Awesome File Type State Usage
YoloV5 28M yolov5 πŸ”₯πŸ”₯πŸ’₯↑ detection βœ… demo
YoloV3 236M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
TinyYoloV3 33M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
YoloV4 176M YOLOv4... πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
SSD 76M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
SSDMobileNetV1 27M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
YoloX 3.5M YOLOX πŸ”₯πŸ”₯new↑ detection βœ… demo
TinyYoloV4VOC 22M yolov4-tiny... πŸ”₯πŸ”₯↑ detection βœ… demo
TinyYoloV4COCO 22M yolov4-tiny... πŸ”₯πŸ”₯↑ detection βœ… demo
YoloR 39M yolor πŸ”₯πŸ”₯new↑ detection βœ… demo
ScaledYoloV4 270M ScaledYOLOv4 πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
EfficientDet 15M ...EfficientDet... πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
EfficientDetD7 220M ...EfficientDet... πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
EfficientDetD8 322M ...EfficientDet... πŸ”₯πŸ”₯πŸ”₯↑ detection βœ… demo
YOLOP 30M YOLOP πŸ”₯πŸ”₯new↑ detection βœ… demo
  • Face Recognition.
Class Size From Awesome File Type State Usage
GlintArcFace 92M insightface πŸ”₯πŸ”₯πŸ”₯↑ faceid βœ… demo
GlintCosFace 92M insightface πŸ”₯πŸ”₯πŸ”₯↑ faceid βœ… demo
GlintPartialFC 170M insightface πŸ”₯πŸ”₯πŸ”₯↑ faceid βœ… demo
FaceNet 89M facenet... πŸ”₯πŸ”₯πŸ”₯↑ faceid βœ… demo
FocalArcFace 166M face.evoLVe... πŸ”₯πŸ”₯πŸ”₯↑ faceid βœ… demo
FocalAsiaArcFace 166M face.evoLVe... πŸ”₯πŸ”₯πŸ”₯↑ faceid βœ… demo
TencentCurricularFace 249M TFace πŸ”₯πŸ”₯↑ faceid βœ… demo
TencentCifpFace 130M TFace πŸ”₯πŸ”₯↑ faceid βœ… demo
CenterLossFace 280M center-loss... πŸ”₯πŸ”₯↑ faceid βœ… demo
SphereFace 80M sphere... πŸ”₯πŸ”₯↑ faceid βœ…οΈ demo
PoseRobustFace 92M DREAM πŸ”₯πŸ”₯↑ faceid βœ…οΈ demo
NaivePoseRobustFace 43M DREAM πŸ”₯πŸ”₯↑ faceid βœ…οΈ demo
MobileFaceNet 3.8M MobileFace... πŸ”₯πŸ”₯↑ faceid βœ… demo
CavaGhostArcFace 15M cavaface... πŸ”₯πŸ”₯↑ faceid βœ… demo
CavaCombinedFace 250M cavaface... πŸ”₯πŸ”₯↑ faceid βœ… demo
MobileSEFocalFace 4.5M face_recog... πŸ”₯πŸ”₯↑ faceid βœ… demo
⚠️ Expand More Details for Lite.AI's Model Zoo.
  • Face Detection.
Class Size From Awesome File Type State Usage
UltraFace 1.1M Ultra-Light... πŸ”₯πŸ”₯πŸ”₯↑ face::detect βœ… demo
RetinaFace 1.6M ...Retinaface πŸ”₯πŸ”₯πŸ”₯↑ face::detect βœ… demo
FaceBoxes 3.8M FaceBoxes πŸ”₯πŸ”₯↑ face::detect βœ… demo
  • Face Alignment.
Class Size From Awesome File Type State Usage
PFLD 1.0M pfld_106_... πŸ”₯πŸ”₯↑ face::align βœ… demo
PFLD98 4.8M PFLD... πŸ”₯πŸ”₯↑ face::align βœ…οΈ demo
MobileNetV268 9.4M ...landmark πŸ”₯πŸ”₯↑ face::align βœ…οΈοΈ demo
MobileNetV2SE68 11M ...landmark πŸ”₯πŸ”₯↑ face::align βœ…οΈοΈ demo
PFLD68 2.8M ...landmark πŸ”₯πŸ”₯↑ face::align βœ…οΈ demo
FaceLandmark1000 2.0M FaceLandm... πŸ”₯↑ face::align βœ…οΈ demo
  • Head Pose Estimation.
Class Size From Awesome File Type State Usage
FSANet 1.2M ...fsanet... πŸ”₯↑ face::pose βœ… demo
  • Face Attributes.
Class Size From Awesome File Type State Usage
AgeGoogleNet 23M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ face::attr βœ… demo
GenderGoogleNet 23M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ face::attr βœ… demo
EmotionFerPlus 33M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ face::attr βœ… demo
VGG16Age 514M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ face::attr βœ… demo
VGG16Gender 512M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ face::attr βœ… demo
SSRNet 190K SSR_Net... πŸ”₯↑ face::attr βœ… demo
EfficientEmotion7 15M face-emo... πŸ”₯↑ face::attr βœ…οΈ demo
EfficientEmotion8 15M face-emo... πŸ”₯↑ face::attr βœ… demo
MobileEmotion7 13M face-emo... πŸ”₯↑ face::attr βœ… demo
ReXNetEmotion7 30M face-emo... πŸ”₯↑ face::attr βœ… demo
  • Classification.
Class Size From Awesome File Type State Usage
EfficientNetLite4 49M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
ShuffleNetV2 8.7M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
DenseNet121 30.7M torchvision πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
GhostNet 20M torchvision πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
HdrDNet 13M torchvision πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
IBNNet 97M torchvision πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
MobileNetV2 13M torchvision πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
ResNet 44M torchvision πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
ResNeXt 95M torchvision πŸ”₯πŸ”₯πŸ”₯↑ classification βœ… demo
  • Segmentation.
Class Size From Awesome File Type State Usage
DeepLabV3ResNet101 232M torchvision πŸ”₯πŸ”₯πŸ”₯↑ segmentation βœ… demo
FCNResNet101 207M torchvision πŸ”₯πŸ”₯πŸ”₯↑ segmentation βœ… demo
  • Style Transfer.
Class Size From Awesome File Type State Usage
FastStyleTransfer 6.4M onnx-models πŸ”₯πŸ”₯πŸ”₯↑ style βœ… demo
  • Colorization.
Class Size From Awesome File Type State Usage
Colorizer 123M colorization πŸ”₯πŸ”₯πŸ”₯↑ colorization βœ… demo
  • Super Resolution.
Class Size From Awesome File Type State Usage
SubPixelCNN 234K ...PIXEL... πŸ”₯↑ resolution βœ… demo

3. Examples for Lite.AI.

More examples can be found at lite.ai-demos. Note that the default backend for Lite.AI is onnxruntime, for the reason that onnxruntime supports the most of onnx's operators. Click the Expand ▢️ button will show you more examples for the specific topic you are interested in.

Example0: Object Detection using YoloV5. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/yolov5s.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_yolov5_1.jpg";
  std::string save_img_path = "../../../logs/test_lite_yolov5_1.jpg";

  auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path); 
  std::vector<lite::cv::types::Boxf> detected_boxes;
  cv::Mat img_bgr = cv::imread(test_img_path);
  yolov5->detect(img_bgr, detected_boxes);
  
  lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
  cv::imwrite(save_img_path, img_bgr);  
  
  delete yolov5;
}

The output is:

Or you can use Newest πŸ”₯πŸ”₯ ! YOLO series's detector YOLOX or YoloR. They got the similar results.


Example1: 1000 Facial Landmarks Detection using FaceLandmarks1000. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/FaceLandmark1000.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_face_landmarks_0.png";
  std::string save_img_path = "../../../logs/test_lite_face_landmarks_1000.jpg";
    
  auto *face_landmarks_1000 = new lite::cv::face::align::FaceLandmark1000(onnx_path);

  lite::cv::types::Landmarks landmarks;
  cv::Mat img_bgr = cv::imread(test_img_path);
  face_landmarks_1000->detect(img_bgr, landmarks);
  lite::cv::utils::draw_landmarks_inplace(img_bgr, landmarks);
  cv::imwrite(save_img_path, img_bgr);
  
  delete face_landmarks_1000;
}

The output is:


Example2: Colorization using colorization. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/eccv16-colorizer.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_colorizer_1.jpg";
  std::string save_img_path = "../../../logs/test_lite_eccv16_colorizer_1.jpg";
  
  auto *colorizer = new lite::cv::colorization::Colorizer(onnx_path);
  
  cv::Mat img_bgr = cv::imread(test_img_path);
  lite::cv::types::ColorizeContent colorize_content;
  colorizer->detect(img_bgr, colorize_content);
  
  if (colorize_content.flag) cv::imwrite(save_img_path, colorize_content.mat);
  delete colorizer;
}

The output is:



Example3: Face Recognition using ArcFace. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ms1mv3_arcface_r100.onnx";
  std::string test_img_path0 = "../../../examples/lite/resources/test_lite_faceid_0.png";
  std::string test_img_path1 = "../../../examples/lite/resources/test_lite_faceid_1.png";
  std::string test_img_path2 = "../../../examples/lite/resources/test_lite_faceid_2.png";

  auto *glint_arcface = new lite::cv::faceid::GlintArcFace(onnx_path);

  lite::cv::types::FaceContent face_content0, face_content1, face_content2;
  cv::Mat img_bgr0 = cv::imread(test_img_path0);
  cv::Mat img_bgr1 = cv::imread(test_img_path1);
  cv::Mat img_bgr2 = cv::imread(test_img_path2);
  glint_arcface->detect(img_bgr0, face_content0);
  glint_arcface->detect(img_bgr1, face_content1);
  glint_arcface->detect(img_bgr2, face_content2);

  if (face_content0.flag && face_content1.flag && face_content2.flag)
  {
    float sim01 = lite::cv::utils::math::cosine_similarity<float>(
        face_content0.embedding, face_content1.embedding);
    float sim02 = lite::cv::utils::math::cosine_similarity<float>(
        face_content0.embedding, face_content2.embedding);
    std::cout << "Detected Sim01: " << sim  << " Sim02: " << sim02 << std::endl;
  }

  delete glint_arcface;
}

The output is:

Detected Sim01: 0.721159 Sim02: -0.0626267


Example4: Face Detection using UltraFace. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ultraface-rfb-640.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_ultraface.jpg";
  std::string save_img_path = "../../../logs/test_lite_ultraface.jpg";

  auto *ultraface = new lite::cv::face::detect::UltraFace(onnx_path);

  std::vector<lite::cv::types::Boxf> detected_boxes;
  cv::Mat img_bgr = cv::imread(test_img_path);
  ultraface->detect(img_bgr, detected_boxes);
  lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
  cv::imwrite(save_img_path, img_bgr);

  delete ultraface;
}

The output is:

⚠️ Expand All Examples for Each Topic in Lite.AI.
3.1 Expand Examples for Object Detection.

3.1 Object Detection using YoloV5. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/yolov5s.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_yolov5_1.jpg";
std::string save_img_path = "../../../logs/test_lite_yolov5_1.jpg";

auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path);
std::vector<lite::cv::types::Boxf> detected_boxes;
cv::Mat img_bgr = cv::imread(test_img_path);
yolov5->detect(img_bgr, detected_boxes);

lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
cv::imwrite(save_img_path, img_bgr);

delete yolov5;
}

The output is:

Or you can use Newest πŸ”₯πŸ”₯ ! YOLO series's detector YOLOX . They got the similar results.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/yolox_s.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_yolox_1.jpg";
  std::string save_img_path = "../../../logs/test_lite_yolox_1.jpg";

  auto *yolox = new lite::cv::detection::YoloX(onnx_path); 
  std::vector<lite::cv::types::Boxf> detected_boxes;
  cv::Mat img_bgr = cv::imread(test_img_path);
  yolox->detect(img_bgr, detected_boxes);
  
  lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
  cv::imwrite(save_img_path, img_bgr);  
  
  delete yolox;
}

The output is:

More classes for general object detection.

auto *detector = new lite::cv::detection::YoloX(onnx_path);  // Newest YOLO detector !!! 2021-07
auto *detector = new lite::cv::detection::YoloV4(onnx_path); 
auto *detector = new lite::cv::detection::YoloV3(onnx_path); 
auto *detector = new lite::cv::detection::TinyYoloV3(onnx_path); 
auto *detector = new lite::cv::detection::SSD(onnx_path); 
auto *detector = new lite::cv::detection::YoloV5(onnx_path); 
auto *detector = new lite::cv::detection::YoloR(onnx_path);  // Newest YOLO detector !!! 2021-05
auto *detector = new lite::cv::detection::TinyYoloV4VOC(onnx_path); 
auto *detector = new lite::cv::detection::TinyYoloV4COCO(onnx_path); 
auto *detector = new lite::cv::detection::ScaledYoloV4(onnx_path); 
auto *detector = new lite::cv::detection::EfficientDet(onnx_path); 
auto *detector = new lite::cv::detection::EfficientDetD7(onnx_path); 
auto *detector = new lite::cv::detection::EfficientDetD8(onnx_path); 
auto *detector = new lite::cv::detection::YOLOP(onnx_path); 
3.2 Expand Examples for Face Recognition.

3.2 Face Recognition using ArcFace. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ms1mv3_arcface_r100.onnx";
  std::string test_img_path0 = "../../../examples/lite/resources/test_lite_faceid_0.png";
  std::string test_img_path1 = "../../../examples/lite/resources/test_lite_faceid_1.png";
  std::string test_img_path2 = "../../../examples/lite/resources/test_lite_faceid_2.png";

  auto *glint_arcface = new lite::cv::faceid::GlintArcFace(onnx_path);

  lite::cv::types::FaceContent face_content0, face_content1, face_content2;
  cv::Mat img_bgr0 = cv::imread(test_img_path0);
  cv::Mat img_bgr1 = cv::imread(test_img_path1);
  cv::Mat img_bgr2 = cv::imread(test_img_path2);
  glint_arcface->detect(img_bgr0, face_content0);
  glint_arcface->detect(img_bgr1, face_content1);
  glint_arcface->detect(img_bgr2, face_content2);

  if (face_content0.flag && face_content1.flag && face_content2.flag)
  {
    float sim01 = lite::cv::utils::math::cosine_similarity<float>(
        face_content0.embedding, face_content1.embedding);
    float sim02 = lite::cv::utils::math::cosine_similarity<float>(
        face_content0.embedding, face_content2.embedding);
    std::cout << "Detected Sim01: " << sim  << " Sim02: " << sim02 << std::endl;
  }

  delete glint_arcface;
}

The output is:

Detected Sim01: 0.721159 Sim02: -0.0626267

More classes for face recognition.

auto *recognition = new lite::cv::faceid::GlintCosFace(onnx_path);  // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::GlintArcFace(onnx_path);  // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::GlintPartialFC(onnx_path); // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::FaceNet(onnx_path);
auto *recognition = new lite::cv::faceid::FocalArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::FocalAsiaArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::TencentCurricularFace(onnx_path); // Tencent(TFace)
auto *recognition = new lite::cv::faceid::TencentCifpFace(onnx_path); // Tencent(TFace)
auto *recognition = new lite::cv::faceid::CenterLossFace(onnx_path);
auto *recognition = new lite::cv::faceid::SphereFace(onnx_path);
auto *recognition = new lite::cv::faceid::PoseRobustFace(onnx_path);
auto *recognition = new lite::cv::faceid::NaivePoseRobustFace(onnx_path);
auto *recognition = new lite::cv::faceid::MobileFaceNet(onnx_path); // 3.8Mb only !
auto *recognition = new lite::cv::faceid::CavaGhostArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::CavaCombinedFace(onnx_path);
auto *recognition = new lite::cv::faceid::MobileSEFocalFace(onnx_path); // 4.5Mb only !
3.3 Expand Examples for Segmentation.

3.3 Segmentation using DeepLabV3ResNet101. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/deeplabv3_resnet101_coco.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_deeplabv3_resnet101.png";
  std::string save_img_path = "../../../logs/test_lite_deeplabv3_resnet101.jpg";

  auto *deeplabv3_resnet101 = new lite::cv::segmentation::DeepLabV3ResNet101(onnx_path, 16); // 16 threads

  lite::cv::types::SegmentContent content;
  cv::Mat img_bgr = cv::imread(test_img_path);
  deeplabv3_resnet101->detect(img_bgr, content);

  if (content.flag)
  {
    cv::Mat out_img;
    cv::addWeighted(img_bgr, 0.2, content.color_mat, 0.8, 0., out_img);
    cv::imwrite(save_img_path, out_img);
    if (!content.names_map.empty())
    {
      for (auto it = content.names_map.begin(); it != content.names_map.end(); ++it)
      {
        std::cout << it->first << " Name: " << it->second << std::endl;
      }
    }
  }
  delete deeplabv3_resnet101;
}

The output is:

More classes for segmentation.

auto *segment = new lite::cv::segmentation::FCNResNet101(onnx_path);
3.4 Expand Examples for Face Attributes Analysis.

3.4 Age Estimation using SSRNet . Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ssrnet.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_ssrnet.jpg";
  std::string save_img_path = "../../../logs/test_lite_ssrnet.jpg";

  lite::cv::face::attr::SSRNet *ssrnet = new lite::cv::face::attr::SSRNet(onnx_path);

  lite::cv::types::Age age;
  cv::Mat img_bgr = cv::imread(test_img_path);
  ssrnet->detect(img_bgr, age);
  lite::cv::utils::draw_age_inplace(img_bgr, age);
  cv::imwrite(save_img_path, img_bgr);
  std::cout << "Default Version Done! Detected SSRNet Age: " << age.age << std::endl;

  delete ssrnet;
}

The output is:

More classes for face attributes analysis.

auto *attribute = new lite::cv::face::attr::AgeGoogleNet(onnx_path);  
auto *attribute = new lite::cv::face::attr::GenderGoogleNet(onnx_path); 
auto *attribute = new lite::cv::face::attr::EmotionFerPlus(onnx_path);
auto *attribute = new lite::cv::face::attr::VGG16Age(onnx_path);
auto *attribute = new lite::cv::face::attr::VGG16Gender(onnx_path);
auto *attribute = new lite::cv::face::attr::EfficientEmotion7(onnx_path); // 7 emotions, 15Mb only!
auto *attribute = new lite::cv::face::attr::EfficientEmotion8(onnx_path); // 8 emotions, 15Mb only!
auto *attribute = new lite::cv::face::attr::MobileEmotion7(onnx_path); // 7 emotions
auto *attribute = new lite::cv::face::attr::ReXNetEmotion7(onnx_path); // 7 emotions
3.5 Expand Examples for Image Classification.

3.5 1000 Classes Classification using DenseNet. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/densenet121.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_densenet.jpg";

  auto *densenet = new lite::cv::classification::DenseNet(onnx_path);

  lite::cv::types::ImageNetContent content;
  cv::Mat img_bgr = cv::imread(test_img_path);
  densenet->detect(img_bgr, content);
  if (content.flag)
  {
    const unsigned int top_k = content.scores.size();
    if (top_k > 0)
    {
      for (unsigned int i = 0; i < top_k; ++i)
        std::cout << i + 1
                  << ": " << content.labels.at(i)
                  << ": " << content.texts.at(i)
                  << ": " << content.scores.at(i)
                  << std::endl;
    }
  }
  delete densenet;
}

The output is:

More classes for image classification.

auto *classifier = new lite::cv::classification::EfficientNetLite4(onnx_path);  
auto *classifier = new lite::cv::classification::ShuffleNetV2(onnx_path); 
auto *classifier = new lite::cv::classification::GhostNet(onnx_path);
auto *classifier = new lite::cv::classification::HdrDNet(onnx_path);
auto *classifier = new lite::cv::classification::IBNNet(onnx_path);
auto *classifier = new lite::cv::classification::MobileNetV2(onnx_path); 
auto *classifier = new lite::cv::classification::ResNet(onnx_path); 
auto *classifier = new lite::cv::classification::ResNeXt(onnx_path);
3.6 Expand Examples for Face Detection.

3.6 Face Detection using UltraFace. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ultraface-rfb-640.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_ultraface.jpg";
  std::string save_img_path = "../../../logs/test_lite_ultraface.jpg";

  auto *ultraface = new lite::cv::face::detect::UltraFace(onnx_path);

  std::vector<lite::cv::types::Boxf> detected_boxes;
  cv::Mat img_bgr = cv::imread(test_img_path);
  ultraface->detect(img_bgr, detected_boxes);
  lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
  cv::imwrite(save_img_path, img_bgr);

  delete ultraface;
}

The output is:

More classes for face detection.

auto *detector = new lite::face::detect::UltraFace(onnx_path);  // 1.1Mb only !
auto *detector = new lite::face::detect::FaceBoxes(onnx_path);  // 3.8Mb only ! 
auto *detector = new lite::face::detect::RetinaFace(onnx_path);  // 1.6Mb only ! CVPR2020
3.7 Expand Examples for Colorization.

3.7 Colorization using colorization. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/eccv16-colorizer.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_colorizer_1.jpg";
  std::string save_img_path = "../../../logs/test_lite_eccv16_colorizer_1.jpg";
  
  auto *colorizer = new lite::cv::colorization::Colorizer(onnx_path);
  
  cv::Mat img_bgr = cv::imread(test_img_path);
  lite::cv::types::ColorizeContent colorize_content;
  colorizer->detect(img_bgr, colorize_content);
  
  if (colorize_content.flag) cv::imwrite(save_img_path, colorize_content.mat);
  delete colorizer;
}

The output is:


3.8 Expand Examples for Head Pose Estimation.

3.8 Head Pose Estimation using FSANet. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/fsanet-var.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_fsanet.jpg";
  std::string save_img_path = "../../../logs/test_lite_fsanet.jpg";

  auto *fsanet = new lite::cv::face::pose::FSANet(onnx_path);
  cv::Mat img_bgr = cv::imread(test_img_path);
  lite::cv::types::EulerAngles euler_angles;
  fsanet->detect(img_bgr, euler_angles);
  
  if (euler_angles.flag)
  {
    lite::cv::utils::draw_axis_inplace(img_bgr, euler_angles);
    cv::imwrite(save_img_path, img_bgr);
    std::cout << "yaw:" << euler_angles.yaw << " pitch:" << euler_angles.pitch << " row:" << euler_angles.roll << std::endl;
  }
  delete fsanet;
}

The output is:

3.9 Expand Examples for Face Alignment.

3.9 1000 Facial Landmarks Detection using FaceLandmarks1000. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/FaceLandmark1000.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_face_landmarks_0.png";
  std::string save_img_path = "../../../logs/test_lite_face_landmarks_1000.jpg";
    
  auto *face_landmarks_1000 = new lite::cv::face::align::FaceLandmark1000(onnx_path);

  lite::cv::types::Landmarks landmarks;
  cv::Mat img_bgr = cv::imread(test_img_path);
  face_landmarks_1000->detect(img_bgr, landmarks);
  lite::cv::utils::draw_landmarks_inplace(img_bgr, landmarks);
  cv::imwrite(save_img_path, img_bgr);
  
  delete face_landmarks_1000;
}

The output is:

More classes for face alignment.

auto *align = new lite::cv::face::align::PFLD(onnx_path);  // 106 landmarks
auto *align = new lite::cv::face::align::PFLD98(onnx_path);  // 98 landmarks
auto *align = new lite::cv::face::align::PFLD68(onnx_path);  // 68 landmarks
auto *align = new lite::cv::face::align::MobileNetV268(onnx_path);  // 68 landmarks
auto *align = new lite::cv::face::align::MobileNetV2SE68(onnx_path);  // 68 landmarks
auto *align = new lite::cv::face::align::FaceLandmark1000(onnx_path);  // 1000 landmarks !
3.10 Expand Examples for Style Transfer.

3.10 Style Transfer using FastStyleTransfer. Download model from Model-Zoo2.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/style-candy-8.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_fast_style_transfer.jpg";
  std::string save_img_path = "../../../logs/test_lite_fast_style_transfer_candy.jpg";
  
  auto *fast_style_transfer = new lite::cv::style::FastStyleTransfer(onnx_path);
 
  lite::cv::types::StyleContent style_content;
  cv::Mat img_bgr = cv::imread(test_img_path);
  fast_style_transfer->detect(img_bgr, style_content);

  if (style_content.flag) cv::imwrite(save_img_path, style_content.mat);
  delete fast_style_transfer;
}

The output is:


3.11 Expand Examples for Image Matting.
  • todo⚠️

4. Lite.AI API Docs.

4.1 Default Version APIs.

More details of Default Version APIs can be found at default-version-api-docs . For examples, the interface for YoloV5 is:

lite::cv::detection::YoloV5

void detect(const cv::Mat &mat, std::vector<types::Boxf> &detected_boxes, 
            float score_threshold = 0.25f, float iou_threshold = 0.45f,
            unsigned int topk = 100, unsigned int nms_type = NMS::OFFSET);
Expand for ONNXRuntime, MNN and NCNN version APIs.

4.2 ONNXRuntime Version APIs.

More details of ONNXRuntime Version APIs can be found at onnxruntime-version-api-docs . For examples, the interface for YoloV5 is:

lite::onnxruntime::cv::detection::YoloV5

void detect(const cv::Mat &mat, std::vector<types::Boxf> &detected_boxes, 
            float score_threshold = 0.25f, float iou_threshold = 0.45f,
            unsigned int topk = 100, unsigned int nms_type = NMS::OFFSET);

4.3 MNN Version APIs.

(todo⚠️: Not implementation now, coming soon.)

lite::mnn::cv::detection::YoloV5

lite::mnn::cv::detection::YoloV4

lite::mnn::cv::detection::YoloV3

lite::mnn::cv::detection::SSD

...

4.4 NCNN Version APIs.

(todo⚠️: Not implementation now, coming soon.)

lite::ncnn::cv::detection::YoloV5

lite::ncnn::cv::detection::YoloV4

lite::ncnn::cv::detection::YoloV3

lite::ncnn::cv::detection::SSD

...

5. Other Docs.

Expand More Details for Other Docs.

5.1 Docs for ONNXRuntime.

5.2 Docs for third_party.

Other build documents for different engines and different targets will be added later.

Library Target Docs
OpenCV mac-x86_64 opencv-mac-x86_64-build.zh.md
OpenCV android-arm opencv-static-android-arm-build.zh.md
onnxruntime mac-x86_64 onnxruntime-mac-x86_64-build.zh.md
onnxruntime android-arm onnxruntime-android-arm-build.zh.md
NCNN mac-x86_64 todo⚠️
MNN mac-x86_64 todo⚠️
TNN mac-x86_64 todo⚠️

6. License.

The code of Lite.AI is released under the MIT License.

7. References.

Many thanks to the following projects. All the Lite.AI's models are sourced from these repos.

  • YOLOX (πŸ”₯πŸ”₯new!!↑)
  • insightface (πŸ”₯πŸ”₯πŸ”₯↑)
  • yolov5 (πŸ”₯πŸ”₯πŸ’₯↑)
Expand More Details for References.

Star πŸŒŸπŸ‘†πŸ» this repo if it does any helps to you ~ πŸ™ƒπŸ€ͺπŸ€

About

Lite.AI πŸš€πŸš€πŸŒŸ is a user friendly C++ lib of awesome AI models. such as YOLOXπŸ”₯, YoloRπŸ”₯, YOLOP, YoloV5, YoloV4, DeepLabV3, ArcFace, CosFace, RetinaFace, SSD, etc.

License:MIT License


Languages

Language:C++ 97.0%Language:CMake 2.7%Language:C 0.3%Language:Shell 0.0%