iiLaurens

lazygit

simple terminal UI for git commands

LLaMA-Factory

Unified Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

graphrag

A modular graph-based Retrieval-Augmented Generation (RAG) system

haystack

:mag: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

MinerU

A one-stop, open-source, high-quality data extraction tool, supports PDF/webpage/e-book extraction.一站式开源高质量数据提取工具，支持PDF/网页/多格式电子书提取。

FlexiGen

Running large language models on a single GPU for throughput-oriented scenarios.

outlines

Structured Text Generation

PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

gotenberg

A developer-friendly API for converting numerous document formats into PDF files, and more!

sglang

SGLang is a fast serving framework for large language models and vision language models.

aim

Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.

lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Liger-Kernel

Efficient Triton Kernels for LLM Training

transformer-explainer

Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization

promptbench

A unified evaluation framework for large language models

langroid

Harness LLMs with Multi-Agent Programming

curl_cffi

Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.

usearch

Fast Open-Source Search & Clustering engine × for Vectors & 🔜 Strings × in C++, C, Python, JavaScript, Rust, Java, Objective-C, Swift, C#, GoLang, and Wolfram 🔍

lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

bypass-paywalls-firefox-clean

infinity

Infinity is a high-throughput, low-latency REST API for serving text-embeddings, reranking models and clip

thepipe

Extract clean data from anywhere, powered by vision-language models ⚡

Efficient-LLMs-Survey

[TMLR 2024] Efficient Large Language Models: A Survey

MInference

[NeurIPS'24 Spotlight] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

dom-to-semantic-markdown

DOM to Semantic-Markdown for use with LLMs

prompt-lookup-decoding

puncc

👋 Puncc is a python library for predictive uncertainty quantification using conformal prediction.

INTERS

This is the repository for our paper "INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning"

pyftpsync

Synchronize directories using FTP(S), SFTP, or file system access.

formatspread

Code accompanying "How I learned to start worrying about prompt formatting".