simonsanvil

llama.cpp

LLM inference in C/C++

Promptify

Prompt Engineering | Prompt Versioning | Use GPT or other prompt based models to get structured output. Join our discord for Prompt-Engineering, LLMs and other latest research

lago

Open Source Metering and Usage Based Billing API ⭐️ Consumption tracking, Subscription management, Pricing iterations, Payment orchestration & Revenue analytics

outlines

Structured Text Generation

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

awesome-public-real-time-datasets

A list of publicly available datasets with real-time data maintained by the team at bytewax.io

text-generation-inference

Large Language Model Text Generation Inference

ensemble-instruct

codebase release for EMNLP2023 paper publication

instructlab

InstructLab Command-Line Interface. Use this to chat with a model and execute the InstructLab workflow to train a model using custom taxonomy data.

LLMLingua

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

LookaheadDecoding

dspy-redteam

Red-Teaming Language Models with DSPy

starlark

Starlark Language

logfire

Uncomplicated Observability for Python and beyond! 🪵🔥

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

txtai

💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

cognita

RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

prometheus

[ICLR 2024 & NeurIPS 2023 WS] An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on a customized score rubric, Prometheus is a good alternative for human evaluation and GPT-4 evaluation.

pytype

A static type analyzer for Python code

text-generation-inference

IBM development fork of https://github.com/huggingface/text-generation-inference

argilla

Argilla is a collaboration platform for AI engineers and domain experts that require high-quality outputs, full data ownership, and overall efficiency.

llm4regression

Examining how large language models (LLMs) perform across various synthetic regression tasks when given (input, output) examples in their context, without any parameter update

ml-engineering

Machine Learning Engineering Open Book

mlx

MLX: An array framework for Apple silicon

distilabel

⚗️ distilabel is a framework for synthetic data and AI feedback for AI engineers that require high-quality outputs, full data ownership, and overall efficiency.

unsloth

Finetune Llama 3, Mistral & Gemma LLMs 2-5x faster with 80% less memory

spacy-llm

🦙 Integrating LLMs into structured NLP pipelines

presidio-research

This package features data-science related tasks for developing new recognizers for Presidio. It is used for the evaluation of the entire system, as well as for evaluating specific PII recognizers or PII detection models.

terraform-ibm-cloud-pak

Terraform modules and examples to support installation for IBM Cloud Paks onto OpenShift clusters

cloud-pak-cli

Cloudctl is a command line tool to manage Container Application Software for Enterprises (CASE)