josephyuzb

ollama

Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models.

whisper.cpp

Port of OpenAI's Whisper model in C/C++

llama3

The official Meta Llama 3 GitHub site

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

llama-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.

tpot

A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

mistral-inference

Official inference library for Mistral models

openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

h2o-llmstudio

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://h2oai.github.io/h2o-llmstudio/

cub

[ARCHIVED] Cooperative primitives for CUDA C++. See https://github.com/NVIDIA/cccl

megablocks

most-common-american-idioms

A book created by xiaolai with the help of ChatGPT and its TTS

cloudpan189-go

天翼云盘命令行客户端(CLI)，基于GO语言实现

Chinese-Mixtral

中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）

Qwen-TensorRT-LLM

rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

llmperf-leaderboard

llm-inference-benchmark

LLM Inference benchmark

tensorflow_hmm

A tensorflow implementation of an HMM layer

ROCR-Runtime

ROCm Platform Runtime: ROCr a HPC market enhanced HSA based runtime

omniperf

Advanced Profiling and Analytics for AMD Hardware

vllm-rocm

vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs

FLASHNN

llmperf

rank_dataset

PyTorch Dataset Rank Dataset

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

LLM-System-Requirements

Open-source calculator for LLM system requirements.

llm-inference-benchmark

LLM 推理服务性能测试

tf_benchmarks

A benchmark framework for Tensorflow