tattrongvu

aifoundation-docs

llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

lm-evaluation-harness

A framework for few-shot evaluation of language models.

deepeval

The LLM Evaluation Framework

create-tsi

Create-tsi is a generative AI RAG toolkit which generates AI Applications with low code.

create-llama

The easiest way to get started with LlamaIndex

insanely-fast-whisper

mergekit

Tools for merging pretrained large language models.

megablocks

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

asyncpg

A fast PostgreSQL Database Client Library for Python/asyncio.

S-LoRA

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

LookaheadDecoding

codellama

Inference code for CodeLlama models

yarn

YaRN: Efficient Context Window Extension of Large Language Models

state-of-open-source-ai

:closed_book: Clarity in the current fast-paced mess of Open Source innovation

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference

bark

🔊 Text-Prompted Generative Audio Model

AutoGPT

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://codellama.h2o.ai/

petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

FlexGen

Running large language models on a single GPU for throughput-oriented scenarios.

llm-continuous-batching-benchmarks

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

pytriton

PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.

ray-llm

RayLLM - LLMs on Ray

llama

Inference code for Llama models

rag-stack

🤖 Deploy a private ChatGPT alternative hosted within your VPC. 🔮 Connect it to your organization's knowledge base and use it as a corporate oracle. Supports open-source LLMs like Llama 2, Falcon, and GPT4All.

langflow

⛓️ Langflow is a visual framework for building multi-agent and RAG applications. It's open-source, Python-powered, fully customizable, model and vector store agnostic.