sunshineflg

FlagEmbedding

Retrieval and Retrieval-augmented LLMs

promptbench

A unified evaluation framework for large language models

WikiChat

WikiChat stops the hallucination of large language models by retrieving data from Wikipedia.

MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型，实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。

VisualGLM-6B

Chinese and English multimodal conversational language model | 多模态中英双语对话语言模型

OmniTab

Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

BELLE

BELLE: Be Everyone's Large Language model Engine（开源中文对话大模型）

promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

self-rag

This includes the original implementation of SELF-RAG: Learning to Retrieve, Generate and Critique through self-reflection by Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi.

airllm

AirLLM 70B inference with single 4GB GPU

LLaMA-Factory

A WebUI for Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

LongLoRA

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

ColossalAI

Making large AI models cheaper, faster and more accessible

trlx

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

blog

fastllm

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

eat_pytorch_in_20_days

Pytorch🍊🍉 is delicious, just eat it! 😋😋

Jpom

🚀简而轻的低侵入式在线构建、自动部署、日常运维、项目监控软件

Megatron-LM

Ongoing research training transformer models at scale

BMTools

Tool Learning for Big Models, Open-Source Solutions of ChatGPT-Plugins

flash-attention

Fast and memory-efficient exact attention

SpanBERT

Code for using and evaluating SpanBERT.

Firefly

Firefly: 大模型训练工具，支持训练Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型

fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

ceval

Official github repo for C-Eval, a Chinese evaluation suite for foundation models [NeurIPS 2023]

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

qlora

QLoRA: Efficient Finetuning of Quantized LLMs

alpaca-lora

Instruct-tune LLaMA on consumer hardware

Alpaca-CoT

We unified the interfaces of instruction-tuning data (e.g., CoT data), multiple LLMs and parameter-efficient methods (e.g., lora, p-tuning) together for easy use. We welcome open-source enthusiasts to initiate any meaningful PR on this repo and integrate as many LLM related technologies as possible. 我们打造了方便研究人员上手和使用大模型等微调平台，我们欢迎开源爱好者发起任何有意义的pr！