Predibase

lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

llm_distillation_playbook

Best practices for distilling large language models.

lora_bakeoff

neuropod

A uniform interface to run deep learning models from multiple frameworks

json-mode-benchmark

punica

Serving multiple LoRA finetuned LLM as one

dask-bigquery

dask-sql

Distributed SQL Engine in Python using Dask

langchain

⚡ Building applications with LLMs through composability ⚡

last-successful-commit-action

GitHub action for identifying the last successful commit for a given workflow and branch.

llama_index

LlamaIndex (GPT Index) is a data framework for your LLM applications

PyBump

Bump version in Helm Chart.yaml and setup.py files

server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

huggingface_hub

The official Python client for the Huggingface Hub.

kubernetes-image-puller

Kubernetes Image Puller is used for caching images on a cluster. It creates a DaemonSet downloading and running the relevant container images on each node.

seldon-core

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

volcano

A Cloud Native Batch System (Project under CNCF)

volcano-apis

The API (CRD) of Volcano