songyinghao

llama2.c

Inference Llama 2 in one file of pure C

sentencepiece

Unsupervised text tokenizer for Neural Network-based text generation.

trl

Train transformer language models with reinforcement learning.

wandb

🔥 A tool for visualizing and tracking your machine learning experiments. This repo contains the CLI and Python API.

Yi

A series of large language models trained from scratch by developers @01-ai

TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

lit-gpt

Hackable implementation of state-of-the-art open-source LLMs based on nanoGPT. Supports flash attention, 4-bit and 8-bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

long_llama

LongLLaMA is a large language model capable of handling long contexts. It is based on OpenLLaMA and fine-tuned with the Focused Transformer (FoT) method.

Qwen-Audio

The official repo of Qwen-Audio (通义千问-Audio) chat & pretrained large audio language model proposed by Alibaba Cloud.

data-juicer

A one-stop data processing system to make data higher-quality, juicier, and more digestible for LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大语言模型提供更高质量、更丰富、更易”消化“的数据！

chatllama

ChatLLaMA 📢 Open source implementation for LLaMA-based ChatGPT runnable in a single GPU. 15x faster training process than ChatGPT

bert4torch

An elegent pytorch implement of transformers

test

Measuring Massive Multitask Language Understanding | ICLR 2021

swiss_army_llama

A FastAPI service for semantic text search using precomputed embeddings and advanced similarity measures, with built-in support for various file types through textract.

whisper.api

This project provides an API with user level access support to transcribe speech to text using a finetuned and processed Whisper ASR model.

llama-from-scratch

Llama from scratch, or How to implement a paper without crying

Firefly-LLaMA2-Chinese

Firefly中文LLaMA-2大模型，支持增量预训练Baichuan2、Llama2、Llama、Falcon、Qwen、Baichuan、InternLM、Bloom等大模型

LLaVA-RLHF

Aligning LMMs with Factually Augmented RLHF

LLMChat

A full-stack Webui implementation of Large Language model, such as ChatGPT or LLaMA.

llama2-fine-tune

Scripts for fine-tuning Llama2 via SFT and DPO.

open-chatgpt

The open source implementation of ChatGPT, Alpaca, Vicuna and RLHF Pipeline. 从0开始实现一个ChatGPT.

llama2.c-zh

支持中文场景的的小语言模型 llama2.c-zh

allamo

Simple, hackable and fast implementation for training/finetuning medium-sized LLaMA-based models

open-llama2

从预训练到强化学习的中文llama2

instruct_storyteller_tinyllama2

Training and Fine-tuning an llm in Python and PyTorch.

LLaMA-Rider