Jeffwan

HexGen

Serving LLMs on heterogeneous decentralized clusters.

lorax

Serve 100s of Fine-Tuned LLMs in Production for the Cost of 1

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

kubernetes

Production-Grade Container Scheduling and Management

ai-comic-factory

Generate comic panels using a LLM + SDXL. Powered by Hugging Face 🤗

ai-llm-research

Collect LLM Papers

alpaca-lora

Instruct-tune LLaMA on consumer hardware

awesome-ai-playground

babyagi

containerd

An open and reliable container runtime

DAIL-SQL

A efficient and effective few-shot NL2SQL method on GPT-4.

DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

enhancements

Enhancements tracking repo for Kubernetes

evals

Evals is a framework for evaluating OpenAI models and an open-source registry of benchmarks.

FastChat

The release repo for "Vicuna: An Open Chatbot Impressing GPT-4"

go-runc

runc bindings for Go

helm

Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110).

kind-with-gpus-examples

kuberay

Contributed modules to ray

langchain

⚡ Building applications with LLMs through composability ⚡

litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

LLMFlow

Easy, Fast, Secure and Cost-Efficient LLM Pipelines to generate GhatGPT-like private domain models and knowledgeable agents for your organization.

lm-evaluation-harness

A framework for few-shot evaluation of autoregressive language models.

MOSS

MOSS is an artificial intelligence character in the movie "The Wandering Earth." s, enabling it to autonomously analyze, infer, and make decisions.

OpenAgents

OpenAgents: An Open Platform for Language Agents in the Wild

runc

CLI tool for spawning and running containers according to the OCI specification

spegel

Stateless cluster local OCI registry mirror.

tiktok-opensdk-ios

The TikTok OpenSDK features Login Kit and Share Kit which allow your users to log in using their TikTok account and share content from your app to TikTok.

veTurboIO

A library developed by Volcano Engine for high-performance reading and writing of PyTorch model files.

website

Kubernetes website and documentation repo: