SuperXiang

Active-IT

Code for our EMNLP-2023 paper: "Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks"

Awesome-LLM

Awesome-LLM: a curated list of Large Language Model

be_great

A novel approach for synthesizing tabular data using pretrained large language models

bertviz

BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

Collinear-Constrained-Attention

deep-language-networks

We view Large Language Models as stochastic language layers in a network, where the learnable parameters are the natural language prompts at each layer. We stack two such layers, feeding the output of one layer to the next. We call the stacked architecture a Deep Language Network - DLN

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

faiss_tips

Some useful tips for faiss

FlagEmbedding

Open-source Embeddings

flash-attention

Fast and memory-efficient exact attention

human-eval

Code for the paper "Evaluating Large Language Models Trained on Code"

IncognitoPilot

An AI code interpreter for sensitive data, powered by GPT-4 or Llama 2.

lagent

A lightweight framework for building LLM-based agents

Lion

Code for "Lion: Adversarial Distillation of Proprietary Large Language Models (EMNLP 2023)"

LLaMA-Efficient-Tuning

Easy-to-use LLM fine-tuning framework (LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, ChatGLM2)

LLM-Shearing

Preprint: Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

LM-RMT

Recurrent Memory Transformer

MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型，实现包括二次预训练、有监督微调、奖励建模、强化学习训练。

milvus

A cloud-native vector database, storage for next generation AI applications

OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

sentence-transformers

Multilingual Sentence & Image Embeddings with BERT

SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

sqlcoder

SoTA LLM for converting natural language questions to SQL queries

SupContrast

PyTorch implementation of "Supervised Contrastive Learning" (and SimCLR incidentally)

SuperAdapters

Finetune ALL LLMs with ALL Adapeters on ALL Platforms!

tutel

Tutel MoE: An Optimized Mixture-of-Experts Implementation

WizardLM

Family of instruction-following LLMs powered by Evol-Instruct: WizardLM, WizardCoder and WizardMath

yarn

YaRN: Efficient Context Window Extension of Large Language Models

LLMs-cookbook

Examples and guides for using the LLMs

LongLoRA

Code and documents of LongLoRA and LongAlpaca