shahules786

LLaMA-Factory

Unify Efficient Fine-Tuning of 100+ LLMs

axolotl

Go ahead and axolotl questions

streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

FlagEmbedding

Retrieval and Retrieval-augmented LLMs

openchat

OpenChat: Advancing Open-source Language Models with Imperfect Data

OpenCopilot

🤖 🔥 Language-to-actions engine

LongLoRA

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

deepeval

The LLM Evaluation Framework

1password-teams-open-source

Get a free 1Password Teams membership for your open source project

fastmoe

A fast MoE impl for PyTorch

OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

mixture-of-experts

PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

punica

Serving multiple LoRA finetuned LLM as one

SPIN

The official implementation of Self-Play Fine-Tuning (SPIN)

awesome-mixture-of-experts

A collection of AWESOME things about mixture-of-experts

megablocks

vec2text

utilities for decoding deep representations (like sentence embeddings) back to text

zeno-build

Build, evaluate, understand, and fix LLM-based apps

Awesome-Mixture-of-Experts-Papers

A curated reading list of research in Mixture-of-Experts(MoE).

MS-AMP

Microsoft Automatic Mixed Precision Library

pykoi-rlhf-finetuned-transformers

pykoi: Active learning in one unified interface

AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard

st-moe-pytorch

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

MoEBERT

This PyTorch package implements MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (NAACL 2022).

LLM-SLERP-Merge

Spherical Merge Pytorch/HF format Language Models with minimal feature loss.

MDEL

Multi-Domain Expert Learning

soft-mixture-of-experts

PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)

decontamination

This repository contains code for cleaning your training data of benchmark data to help combat data snooping.

smear

spade-experiments

Experiments to assess SPADE on different LLM pipelines.