inference-engine

There are 19 repositories under inference-engine topic.

FedML-AI / FedML
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.
federated-learning deep-learning distributed-training edge-ai machine-learning on-device-training inference-engine mlops model-deployment model-serving ai-agent
Language:Python 4073
hyperjumptech / grule-rule-engine
Rule engine implementation in Golang
golang hacktoberfest hacktoberfest2021 inference-engine rule rule-based rule-based-engine rule-engine
Language:Go 2041
zjhellofss / KuiperInfer
带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step
inference inference-engine deep-learning deep-neural-networks convolution relu sigmoid graph-algorithms maxpooling caffe pnnx pytorch ncnn diy resnet yolo yolov5
Language:C++ 2026
janhq / cortex
Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM). Powers 👋 Jan
gguf llama2 llamacpp tensorrt-llm accelerated ai inference-engine openai-api stable-diffusion cuda llama llm llms
Language:C++ 1635
onediff
siliconflow / onediff
OneDiff: An out-of-the-box acceleration library for diffusion models.
comfyui diffusers pytorch sdxl stable-diffusion aigc-serving comfyui-workflow cuda inference-engine lcm lcm-lora performance-optimization sd-webui stable-video-diffusion diffusion-models lora sdxl-turbo
Language:Python 1238
Tencent / FeatherCNN
FeatherCNN is a high performance inference engine for convolutional neural networks.
convolutional-neural-networks inference-engine caffe android ios arm-neon
Language:C++ 1209
PaddlePaddle / Paddle.js
Paddle.js is a web project for Baidu PaddlePaddle, which is an open source deep learning framework running in the browser. Paddle.js can either load a pre-trained model, or transforming a model from paddle-hub with model transforming tools provided by Paddle.js. It could run in every browser with WebGL/WebGPU/WebAssembly supported. It could also run in Baidu Smartprogram and WX miniprogram.
webgl webgpu webassembly model inference-engine paddlepaddle ocr deep-learning
Language:JavaScript 946
Adlik / Adlik
Adlik: Toolkit for Accelerating Deep Learning Inference
deep-learning inference tensorflow-serving openvino tensorrt compiler inference-engine model-optimizer docker-images
Language:C++ 789
msnh2012 / Msnhnet
🔥 (yolov3 yolov4 yolov5 unet ...)A mini pytorch inference framework which inspired from darknet.
yolov3 yolov4 yolov5 pytorch inference-engine darknet jetson-nx mobilenetv2 mobilenetyolo
Language:C++ 728
PygmalionAI / aphrodite-engine
PygmalionAI's large-scale inference engine
api-rest avx512 cuda inference-engine inferentia machine-learning rocm
Language:Python 598
Forward
Tencent / Forward
A library for high performance deep learning inference on NVIDIA GPUs.
deep-learning forward tensorrt tensorflow pytorch keras neural-network gpu inference-engine inference cuda onnx
Language:C++ 546
PaddlePaddle / Anakin
High performance Cross-platform Inference-engine, you could run Anakin on x86-cpu,arm, nv-gpu, amd-gpu,bitmain and cambricon devices.
ai inference-engine arm nvidia amd intel bitmain cambricon high-performance cross-platform
Language:C++ 530
HoloClean / holoclean
A Machine Learning System for Data Enrichment.
data-enrichment data-science inference-engine machine-learning pytorch
Language:Python 511
pylint-dev / astroid
A common base representation of python source code for pylint and other projects
ast parser inference-engine closember hacktoberfest static-code-analysis static-analysis
Language:Python 511
insight-platform / Savant
Python Computer Vision & Video Analytics Framework With Batteries Included
computer-vision deepstream edge-computing inference-engine machine-learning nvidia-deepstream-sdk deep-learning nvidia object-detection video cuda opencv tensorrt instance-segmentation peoplenet yolo yolov5-face yolov8 yolov8-face
Language:Python 484
ulfurinn / wongi-engine
A rule engine written in Ruby.
ruby rule-engine rete inference-engine
Language:Ruby 481
buguroo / pyknow
PyKnow: Expert Systems for Python
python3 expert-system inference-engine
Language:Python 460
gottingen / search-legend
docs for search system and ai infra
ai performance search-engine deep-learning neural-network python inference-engine tensorflow tensorflow2
Language:C++ 236
quic / ai-hub-models
The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
deeplearning demos inference inference-api inference-engine machine-learning machinelearning onnx pytorch qnn tensorflow-lite
Language:Python 231
BMW-InnovationLab / BMW-TensorFlow-Inference-API-CPU
This is a repository for an object detection inference API using the Tensorflow framework.
tensorflow inference api cpu deep-learning object-detection computer-vision detection-inference-api docker tensorflow-framework predictions bounding-boxes docker-image docker-container deeplearning computervision docker-ce inference-engine inference-server rest-api
Language:Python 186
MIVisionX
ROCm / MIVisionX
MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX also delivers a highly optimized open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions.
amd-opencl amd-opencv amd-openvx computer-vision inference inference-engine khronos-openvx machine-learning neural-network nnef onnx opencl openvx openvx-extensions openvx-neural-network rocm ryzen virtual-reality windows-machine-learning winml
Language:C++ 181
midea-ai / Aidget
Ai edge toolbox，专门面向边端设备尤其是嵌入式RTOS平台，AI模型部署工具链，包括模型推理引擎和模型压缩工具
inference inference-engine mcu pruning rtos simd tflm resrep asr wakeup dsp deep-learning hifi5
Language:Python 151
HolmesShuan / CNN-Inference-Engine-Quick-View
A quick view of high-performance convolution neural networks (CNNs) inference engines on mobile devices.
ai-benchmarks cnn cnn-inference-engine cnns inference-engine inference-engines speed-benchmarks
150
haobosang / TinyTensor
TinyTensor is a tool for running already trained NN (Neural Network) models to be able to use them for inference of various tasks such as image classification, semantic segmentation, etc.
cpp deep-learning inference-engine neural-network
Language:C++ 145
nilp0inter / experta
Expert Systems for Python
knowledge-base inference inference-engine clips python3
Language:Python 140
openvino_contrib
openvinotoolkit / openvino_contrib
Repository for OpenVINO's extra modules
openvino pytorch arm java inference-engine nvidia-gpu
Language:C++ 97
solidglue / Recommender_System_Inference_Services
Large scale recommender system inference Microservices and APIs （Dubbo 、gRPC and REST ） with Golang.
api deep-learning inference-engine machine-learning microservices recommender-systems
Language:Go 96
nrl-ai / daisykit
DaisyKit is an easy AI toolkit with face mask detection, pose detection, background matting, barcode detection, face recognition and more. - with NCNN, OpenCV, Python wrappers
mobile computer-vision embedded inference-engine cpp deployment ncnn vulkan no-code python face-detection background-matting deep-learning face-mask-detection machine-learning barcode-detection hand-pose object-detection neural-network
Language:C++ 95
deep-vision-processing
cansik / deep-vision-processing
Deep computer-vision algorithms for the Processing framework.
deep-neural-networks computer-vision processing pose-estimation machine-learning classification inference-engine cuda-support
Language:Java 88
Torsion-Audio / nn-inference-template
Neural network inference template for real-time cricital audio environments - presented at ADC23
audio audio-plugin deep-learning inference inference-engine juce libtorch machine-learning neural-network onnx onnxruntime pytorch tensorflow tensorflow-lite
Language:C++ 86
HoloClean / HoloClean-Legacy-deprecated
A Machine Learning System for Data Enrichment.
machine-learning inference-engine pytorch data-science data-enrichment data-cleaning
Language:Python 75
T-head-Semi / csi-nn2
An optimized neural network operator library for chips base on Xuantie CPU.
deep-learning inference-engine neural-network risc-v riscv riscv-assembly
Language:C 74
BMW-InnovationLab / BMW-IntelOpenVINO-Detection-Inference-API
This is a repository for a No-Code object detection inference API using the OpenVINO. It's supported on both Windows and Linux Operating systems.
openvino deeplearning inference cpu docker neural-network nocode object-detection detection-api inference-engine resnet detection-algorithm rest-api computer-vision openvino-toolkit openvino-model-zoo
Language:Python 72
CoderLSF / fast-llama
Runs LLaMA with Extremely HIGH speed
cpu-inference inference-engine llama llama2
Language:C++ 71
Media-Smart / cheetahinfer
A C++ inference SDK based on TensorRT
cpp inference-engine tensorrt
Language:C++ 69
RubixML / Server
A standalone inference server for trained Rubix ML estimators.
machine-learning http-server model-server infrastructure api model-deployment microservice json-api php rest-api rubix-ml inference inference-engine php-ml ml-infrastructure php-machine-learning inference-server rubix-server
Language:PHP 61

inference-engine

FedML-AI / FedML

hyperjumptech / grule-rule-engine

zjhellofss / KuiperInfer

janhq / cortex

siliconflow / onediff

Tencent / FeatherCNN

PaddlePaddle / Paddle.js

Adlik / Adlik

msnh2012 / Msnhnet

PygmalionAI / aphrodite-engine

Tencent / Forward

PaddlePaddle / Anakin

HoloClean / holoclean

pylint-dev / astroid

insight-platform / Savant

ulfurinn / wongi-engine

buguroo / pyknow

gottingen / search-legend

quic / ai-hub-models

BMW-InnovationLab / BMW-TensorFlow-Inference-API-CPU

ROCm / MIVisionX

midea-ai / Aidget

HolmesShuan / CNN-Inference-Engine-Quick-View

haobosang / TinyTensor

nilp0inter / experta

openvinotoolkit / openvino_contrib

solidglue / Recommender_System_Inference_Services

nrl-ai / daisykit

cansik / deep-vision-processing

Torsion-Audio / nn-inference-template

HoloClean / HoloClean-Legacy-deprecated

T-head-Semi / csi-nn2

BMW-InnovationLab / BMW-IntelOpenVINO-Detection-Inference-API

CoderLSF / fast-llama

Media-Smart / cheetahinfer

RubixML / Server