inference-optimization

There are 4 repositories under inference-optimization topic.

google / XNNPACK
High-efficiency floating-point neural network inference operators for mobile, server, and Web
neural-networks inference inference-optimization simd cpu multithreading matrix-multiplication convolutional-neural-networks convolutional-neural-network neural-network mobile-inference
Language:C 1699
alibaba / BladeDISC
BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.
compiler deep-learning inference-optimization machine-learning mlir neural-network pytorch tensorflow
Language:C++ 745
jiazhihao / TASO
The Tensor Algebra SuperOptimizer for Deep Learning
deep-learning deep-neural-networks inference-optimization
Language:C++ 671
Oulu-IMEDS / pytorch_bn_fusion
Batch normalization fusion for PyTorch
batch-normalization deep-learning deep-neural-networks inference-optimization pytorch
Language:Python 194
mit-han-lab / inter-operator-scheduler
[MLSys 2021] IOS: Inter-Operator Scheduler for CNN Acceleration
acceleration cnn inference-optimization parallelism
Language:C++ 186
ZFTurbo / Keras-inference-time-optimizer
Optimize layers structure of Keras model to reduce computation time
keras inference-optimization
Language:Python 157
Rapternmn / PyTorch-Onnx-Tensorrt
A set of tool which would make your life easier with Tensorrt and Onnxruntime. This Repo is designed for YoloV3
darknet inference-optimization onnx onnx-torch onnxruntime pytorch tensorrt yolov3
Language:Python 80
lmaxwell / Armednn
cross-platform modular neural network inference library, small and efficient
inference-engine neural-network eigen eigen3 lstm conv1d inference-optimization
Language:C++ 13
keli-wen / AGI-Study
The blog, read report and code example for AGI/LLM related knowledge.
code-examples demo inference-optimization llm train
Language:Python 5
Bisonai / ncnn
Modified inference engine for quantized convolution using product quantization
edge-machine-learning quantization product-quantization mobile-deep-learning inference-optimization inference-acceleration
Language:C++ 4
effrosyni-papanastasiou / constrained-em
A constrained expectation-maximization algorithm for feasible graph inference.
expectation-maximization network-inference feasibility expectation-maximisation-algorithm inference-optimization
Language:Jupyter Notebook 4
batch-partitioning
sjlee25 / batch-partitioning
Batch Partitioning for Multi-PE Inference with TVM (2020)
tvm deep-learning data-parallelism dl-compiler dl-optimization inference-optimization
Language:Python 3
zhliuworks / Fast-MobileNetV2
🤖️ Optimized CUDA Kernels for Fast MobileNetV2 Inference
cuda-kernels inference-optimization mobilenet-v2
Language:Cuda 3
kiritigowda / mivisionx-inference-analyzer
MIVisionX Python Inference Analyzer uses pre-trained ONNX/NNEF/Caffe models to analyze inference results and summarize individual image results
openvx mivisionx onnx nnef caffe inference inference-engine inference-optimization amd amdgpu opencl rocm inceptionv4 resnet vgg resnet-50 squeezenet mivisionx-inference-analyzer docker-images nnir
Language:Python 2
ksm26 / Efficiently-Serving-LLMs
Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Predibase’s LoRAX framework inference server.
batch-processing deep-learning-techniques inference-optimization machine-learning-operations model-acceleration model-inference-service model-serving optimization-techniques performance-enhancement scalability-strategies server-optimization text-generation large-scale-deployment serving-infrastructure
Language:Jupyter Notebook 2
aalbaali / LieBatch
Batch estimation on Lie groups
inference-optimization lie-groups batch-optimization g2o state-estimation
Language:MATLAB 1
cedrickchee / pytorch-mobile-ios
PyTorch Mobile: iOS examples
edge-ai inference-optimization ios-app libtorch machine-learning pytorch-mobile
Language:Swift 1
piotrostr / infer-trt
Interface for TensorRT engines inference along with an example of YOLOv4 engine being used.
deep-learning inference-optimization object-detection tensorrt
Language:Python 1
Wb-az / YOLOv8-Image-detection
YOLOV8 - Object detection
average-precision inference-optimization live-streaming object-detection openvino-inference-engine openvino-toolkit optimization-algorithms pandas pytorch ray-tune ultralytics yolov8 computer-vision deep-learning
Language:Jupyter Notebook 1
ankdeshm / inference-optimization
A compilation of various ML and DL models and ways to optimize the their inferences.
acceleration-model cnn-classification cuda cudf cuml inference-optimization nvidia-rapids python3 pytorch quantization torchscript xgboost
Language:Jupyter Notebook 0
goshaQ / inference-optimizer
A simple tool that applies structure-level optimizations (e.g. Quantization) to a TensorFlow model
tensorflow tensorflow-models inference-optimization
Language:Python 0
prabhath-r / Enhancing-BERT-for-NLP-Tasks
Improving Natural Language Processing tasks using BERT-based models
bert-fine-tuning inference-optimization transformers
Language:Jupyter Notebook 0
cedrickchee / pytorch-mobile-android
PyTorch Mobile: Android examples of usage in applications
android-app edge-ai inference-optimization libtorch machine-learning pytorch-mobile
Language:Java
ieee820 / ncnn
ncnn is a high-performance neural network inference framework optimized for the mobile platform
inference-optimization mobile-networks ncnn arm-neon
Language:C++

inference-optimization

google / XNNPACK

alibaba / BladeDISC

jiazhihao / TASO

Oulu-IMEDS / pytorch_bn_fusion

mit-han-lab / inter-operator-scheduler

ZFTurbo / Keras-inference-time-optimizer

Rapternmn / PyTorch-Onnx-Tensorrt

lmaxwell / Armednn

keli-wen / AGI-Study

Bisonai / ncnn

effrosyni-papanastasiou / constrained-em

sjlee25 / batch-partitioning

zhliuworks / Fast-MobileNetV2

kiritigowda / mivisionx-inference-analyzer

ksm26 / Efficiently-Serving-LLMs

aalbaali / LieBatch

cedrickchee / pytorch-mobile-ios

piotrostr / infer-trt

Wb-az / YOLOv8-Image-detection

ankdeshm / inference-optimization

goshaQ / inference-optimizer

prabhath-r / Enhancing-BERT-for-NLP-Tasks

cedrickchee / pytorch-mobile-android

ieee820 / ncnn