zpye / SimpleInfer

A simple neural network inference framework

SimpleInfer

SimpleInfer is a neural network inference framework based on KuiperInfer.

Build

SimpleInfer uses xmake to build library and tests.

git clone --recursive https://github.com/zpye/SimpleInfer.git
cd SimpleInfer
xmake config -a x64 -m release
xmake -w --all

Run

After building successfully, run test-yolo to check.

xmake run test-yolo

YOLO Result

Here are visualized results of YOLO detection.

Working With Python

1. Set environment PYTHON_ROOT where python binary exists, pybind11 needs ${PYTHON_ROOT}/include and ${PYTHON_ROOT}/libs for compiling and linking.

2. Set --build_python=true after xmake config and build:

xmake config -a x64 -m release --build_python=true
xmake -w --all

3. install package by pip:

pip install build/python/

4. run python test:

python test/test_python/test_model.py

(Experimental) Halide Programming

Note: Only upsample nearest layer has an implementation of Halide.

1. Set environment HALIDE_ROOT for Halide installation path, using release packages from https://github.com/halide/Halide/releases is a good choice.

2. Set --halide=true after xmake config and build:

xmake config -a x64 -m release --halide=true
xmake build halide_layers
xmake -w --all

Reference

KuiperInfer -> basic framework

ncnn, simpleocv -> pnnx ir, simpleocv and mat-pixel operations

Eigen, tensorflow -> Eigen tensor

abseil -> logging, string format operations

CGraph -> graph based pipeline

highway, Simd -> SIMD, GEMM, Winograd, parallel

benchmark, Catch2 -> benchmark and unit tests

pybind11 -> python bindings of c++

stb -> image loader and image writer

Halide -> Halide programming

tmp -> pnnx models