inference

There are 17 repositories under inference topic.

google / mediapipe
Cross-platform, customizable ML solutions for live and streaming media.
mediapipe c-plus-plus computer-vision deep-learning android video-processing audio-processing mobile-development machine-learning inference graph-framework graph-based calculator framework pipeline-framework stream-processing perception
Language:C++ 17367
Tencent / ncnn
ncnn is a high-performance neural network inference framework optimized for the mobile platform
android arm-neon artificial-intelligence caffe darknet deep-learning high-preformance inference ios keras mlir mxnet ncnn neural-network onnx pytorch riscv simd tensorflow vulkan
Language:C++ 14641
aws / amazon-sagemaker-examples
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
sagemaker aws reinforcement-learning machine-learning deep-learning examples jupyter-notebook mlops data-science training inference
Language:Jupyter Notebook 6921
microsoft / DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
deep-learning pytorch gpu machine-learning billion-parameters data-parallelism model-parallelism inference pipeline-parallelism
Language:Python 6798
Linzaer / Ultra-Light-Fast-Generic-Face-Detector-1MB
💎1MB lightweight face detection model (1MB轻量级人脸检测模型)
arm face-detection inference mnn ncnn
Language:Python 6367
dusty-nv / jetson-inference
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
caffe computer-vision deep-learning digits embedded image-recognition inference jetson jetson-nano jetson-tx1 jetson-tx2 jetson-xavier jetson-xavier-nx machine-learning nvidia object-detection robotics segmentation tensorrt video-analytics
Language:C++ 5617
gcanti / io-ts
Runtime type system for IO decoding/encoding
typescript validation inference types runtime
Language:TypeScript 5441
triton-inference-server / server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
inference gpu machine-learning deep-learning cloud datacenter edge
Language:Python 3688
ColossalAI
hpcaitech / ColossalAI
Colossal-AI: A Unified Deep Learning System for Big Model Era
ai big-model data-parallelism deep-learning distributed-computing heterogeneous-training hpc inference large-scale model-parallelism pipeline-parallelism
Language:Python 3629
Tencent / TNN
TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts. TNN has been deployed in multiple Apps from Tencent, such as Mobile QQ, Weishi, Pitu, etc. Contributions are welcome to work in collaborative with us and make TNN a better framework.
deep-learning mnn ncnn inference pytorch tensorflow coreml tensorrt tengine openvino face-detection hairsegmentaion ocr
Language:C++ 3519
NVIDIA-AI-IOT / torch2trt
An easy to use PyTorch to TensorRT converter
classification inference jetson-nano jetson-tx2 jetson-xavier pytorch tensorrt
Language:Python 3351
openvinotoolkit / openvino
OpenVINO™ Toolkit repository
inference-engine inference deep-learning performance openvino model-optimizer
Language:C++ 3283
typedb
vaticle / typedb
TypeDB: a strongly-typed database
database enterprise-knowledge-graph graph graph-database graph-theory graph-visualisation graphdb hyper-relational inference knowledge-base knowledge-graph knowledge-representation logic reasoning relational strongly-typed type-system typedb typeql
Language:Java 3086
Trusted-AI / adversarial-robustness-toolbox
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
adversarial-attacks adversarial-examples adversarial-machine-learning ai artificial-intelligence attack blue-team evasion extraction inference machine-learning poisoning privacy python red-team trusted-ai trustworthy-ai
Language:Python 3012
ts-pattern
gvergnaud / ts-pattern
🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference.
branching conditions exhaustive inference javascript matching pattern pattern-matching ts type-inference typescript
Language:TypeScript 2722
bytedance / lightseq
LightSeq: A High Performance Library for Sequence Processing and Generation
accelerate bart beam-search bert cuda diverse-decoding gpt inference multilingual-nmt sampling training transformer
Language:Cuda 2142
tobegit3hub / tensorflow_template_application
TensorFlow template application for deep learning
tensorflow tfrecords libsvm csv deep-learning machine-learning mlp cnn lstm inference tensorboard wide-and-deep serving
Language:Python 1863
HuaizhengZhang / Awesome-System-for-Machine-Learning
A curated list of research in machine learning systems (MLSys). Paper notes are also provided.
machine-learning system deep-learning optimization paper resouce-management deep-reinforcement-learning edge-computing computer-system distributed-systems infrastructure deep-neural-networks automl model-database inference
1641
Maratyszcza / NNPACK
Acceleration package for neural networks on multi-core CPUs
neural-network neural-networks convolutional-layers inference high-performance high-performance-computing simd cpu multithreading fast-fourier-transform winograd-transform matrix-multiplication
Language:C 1557
Delta-ML / delta
DELTA is a deep learning based natural language and speech processing platform.
nlp deep-learning tensorflow speech sequence-to-sequence seq2seq speech-recognition text-classification speaker-verification nlu text-generation emotion-recognition tensorflow-serving tensorflow-lite inference asr serving front-end custom-ops ops
Language:Python 1514
budgetml
ebhy / budgetml
Deploy a ML inference service on a budget in less than 10 lines of code.
api data-science deployment fastapi inference machine-learning mlops
Language:Python 1268
Tencent / TurboTransformers
a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.
nlp transformer bert decoder gpu machine-translation inference huggingface-transformers pytorch albert roberta gpt2
Language:C++ 1163
nebuly-ai / nebullvm
Easy-to-use library to boost AI inference leveraging state-of-the-art optimization techniques.
compiler computing deep-learning huggingface inference machine-learning onnx openvino pytorch tensorflow tensorrt tvm
Language:Jupyter Notebook 1146
google / XNNPACK
High-efficiency floating-point neural network inference operators for mobile, server, and Web
neural-networks inference inference-optimization simd cpu multithreading matrix-multiplication convolutional-neural-networks convolutional-neural-network neural-network mobile-inference
Language:C 1142
kamalkraj / BERT-NER
Pytorch-Named-Entity-Recognition-with-BERT
bert named-entity-recognition pytorch conll-2003 cpp11 bert-ner inference curl postman pretrained-models
Language:Python 1035
tucan9389 / awesome-ml-demos-with-ios
The challenge projects for Inferencing machine learning models on iOS
ios machine-learning coreml mlkit tensorflow tensorflow-lite demo awesome inference
Language:Python 922
transformer-deploy
ELS-RD / transformer-deploy
Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
inference deep-learning natural-language-processing deployment machine-learning server
Language:Python 915
uber / neuropod
A uniform interface to run deep learning models from multiple frameworks
tensorflow pytorch keras deep-learning deeplearning machine-learning machinelearning inference incubation
Language:C++ 904
awslabs / multi-model-server
Multi Model Server is a tool for serving neural net models for inference
mxnet deep-learning inference ai neural-network onnx server
Language:Java 874
FenTechSolutions / CausalDiscoveryToolbox
Package for causal inference in graphs and in the pairwise settings. Tools for graph structure recovery and dependencies are included.
algorithm causal-discovery causal-inference causal-models causality graph graph-structure-recovery inference machine-learning python toolbox
Language:Python 761
huawei-noah / bolt
Bolt is a deep learning library with high performance and heterogeneous flexibility.
bolt inference high-performance x86 mali deep-learning cv nlp arm onnx caffe tensorflow android ios huawei cnn rnn noah mobile
Language:C++ 708
NVIDIA-AI-IOT / tf_trt_models
TensorFlow models accelerated with NVIDIA TensorRT
tensorflow tensorrt nvidia jetson tx1 tx2 models neural-network object-detection image-classification train inference realtime optimize
Language:Python 664
openvino_notebooks
openvinotoolkit / openvino_notebooks
📚 A collection of Jupyter notebooks for learning and experimenting with OpenVINO 👓
deep-learning machine-learning computer-vision openvino inference
Language:Jupyter Notebook 638
deepsparse
neuralmagic / deepsparse
Neural network inference engine that delivers GPU-class performance for sparsified models on CPUs
auto-ml computer-vision cpu-inference-api cpus deepsparse-engine inference machinelearning ml nlp object-detection onnx pretrained-models pruning pytorch quantization sparsification sparsification-recipe sparsified-models tensorflow yolov3
Language:Python 621
onepanelio / onepanel
The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.
deeplearning pipelines machinelearning jupyterlab tensorflow pytorch tensorboard computer-vision mlops workflows aiops annotation hyperparameter-tuning inference training labeling ai etl
Language:Go 600
csarron / awesome-emdl
Embedded and mobile deep learning research resources
efficient-neural-networks deep-neural-networks deep-learning mobile-ai embedded-ai mobile-inference mobile-deep-learning pruning quantization neural-network-compression inference
592

ezoic increase your site revenue

Related Topics

deep-learning object-detection computer-vision tensorflow pytorch