wns823

llm-sp

Papers and resources related to the security and privacy of LLMs 🤖

ADAS

Automated Design of Agentic Systems

llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

sgcrl

LLM-QAT

Code repo for the paper "LLM-QAT Data-Free Quantization Aware Training for Large Language Models"

torchchat

Run PyTorch LLMs locally on servers, desktop and mobile

hqq

Official implementation of Half-Quadratic Quantization (HQQ)

qllm-eval

Code Repository of Evaluating Quantized Large Language Models

llama-stack

Model components of the Llama Stack APIs

llama-stack-apps

Agentic components of the Llama Stack APIs

llmtools

Finetuning Large Language Models on One Consumer GPU in Under 4 Bits

awesome-model-quantization

A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.

kompute

General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). Blazing fast, mobile-enabled, asynchronous and optimized for advanced GPU data processing usecases. Backed by the Linux Foundation.

opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

persona-hub

Official repo for the paper "Scaling Synthetic Data Creation with 1,000,000,000 Personas"

pyreft

ReFT: Representation Finetuning for Language Models

redco

NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference

Perplexica

Perplexica is an AI-powered search engine. It is an Open source alternative to Perplexity AI

optimum-nvidia

weight-selection

FoFo

unsloth

Finetune Llama 3.2, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

mambaformer-icl

MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248

OSWorld

OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

SWE-agent

SWE-agent takes a GitHub issue and tries to automatically fix it, using GPT-4, or your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges.

Triton-Puzzles

Puzzles for learning Triton

lightning-thunder

Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors at once; across one or thousands of GPUs.

modules

🧩 Official registry of Rivet Modules.

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

transformer-debugger