There are 3 repositories under triton-inference-server topic.
Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
Add bisenetv2. My implementation of BiSeNet
This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
OpenAI compatible API for TensorRT LLM triton backend
【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架,支持dynamic batching、streaming模式,支持python/c++双语言,可限制,可拓展,高性能。帮助用户快速地将模型部署到线上,并通过http/rpc接口方式提供服务。
ClearML - Model-Serving Orchestration and Repository Solution
The Triton backend for the ONNX Runtime.
Deploy stable diffusion model with onnx/tenorrt + tritonserver
NVIDIA-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU
Deploy DL/ ML inference pipelines with minimal extra code.
Compare multiple optimization methods on triton to imporve model service performance
Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NVIDIA-docker/ minIO/ Supervisord on AGX or PC from scratch.
Build Recommender System with PyTorch + Redis + Elasticsearch + Feast + Triton + Flask. Vector Recall, DeepFM Ranking and Web Application.
Diffusion Model for Voice Conversion
Advanced inference pipeline using NVIDIA Triton Inference Server for CRAFT Text detection (Pytorch), included converter from Pytorch -> ONNX -> TensorRT, Inference pipelines (TensorRT, Triton server - multi-format). Supported model format for Triton inference: TensorRT engine, Torchscript, ONNX
Provides an ensemble model to deploy a YoloV8 ONNX model to Triton
Serving Example of CodeGen-350M-Mono-GPTJ on Triton Inference Server with Docker and Kubernetes
C++ application to perform computer vision tasks using Nvidia Triton Server for model inference
This repository is an AI bootcamp material that consist of a workflow for computer vision
A demo of Redis Enterprise as the Online Feature Store deployed on GCP with Feast and NVIDIA Triton Inference Server.
Magface Triton Inferece Server Using Tensorrt
FastAPI middleware for comparing different ML model serving approaches
Deploy KoGPT with Triton Inference Server
Generate Glue Code in seconds to simplify your Nvidia Triton Inference Server Deployments
Miscellaneous codes and writings for MLOps
Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code.
Triton Inference Server Web UI
Triton face detection & recognition
TensorFlow Lite backend with ArmNN delegate support for Nvidia Triton
Provides an ensemble model to deploy a YOLOv8 TensorRT model to Triton
The Purpose of this repository is to create a DeepStream/Triton-Server sample application that utilizes yolov7, yolov7-qat, yolov9 models to perform inference on video files or RTSP streams.
MLModelService wrapping Nvidia's Triton Server