fastertransformer

There are 0 repository under fastertransformer topic.

InternLM / lmdeploy
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
codellama cuda-kernels deepspeed fastertransformer internlm llama llama2 llama3 llm llm-inference turbomind
Language:Python 2612
Curt-Park / serving-codegen-gptj-triton
Serving Example of CodeGen-350M-Mono-GPTJ on Triton Inference Server with Docker and Kubernetes
codegen docker fastertransformer kubernetes triton-inference-server pytorch huggingface-transformers
Language:Python 20
detail-novelist / novelist-triton-server
Deploy KoGPT with Triton Inference Server
fastertransformer gptj huggingface kogpt large-language-models transformers triton triton-inference-server
Language:Shell 14
clam004 / triton-ft-api
tutorial on how to deploy a scalable autoregressive causal language model transformer using nvidia triton server
fastapi fastertransformer gpt huggingface nvidia nvidia-docker nvidia-gpu
Language:Python 5
RajeshThallam / fastertransformer-converter
This repository is a code sample to serve Large Language Models (LLM) on a Google Kubernetes Engine (GKE) cluster with GPUs running NVIDIA Triton Inference Server with FasterTransformer backend.
fastertransformer gke googlecloudplatform inference large-scale-machine-learning llm triton-inference-server
Language:Python

InternLM / lmdeploy