baby-care

build-your-own-x

Master programming by recreating your favorite technologies from scratch.

Natural_Language_Processing_with_Transformers

Natural Language Processing with Transformers 中译本，最权威Transformers教程

opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

how-to-optim-algorithm-in-cuda

how to optimize some algorithm in cuda.

CEPE

[ACL 2024] Long-Context Language Modeling with Parallel Encodings

mmrazor

OpenMMLab Model Compression Toolbox and Benchmark.

EAGLE

Official Implementation of EAGLE-1 and EAGLE-2

LookaheadDecoding

fastllm

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

ntk_alibi

NTK scaled version of ALiBi position encoding in Transformer.

laser

The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction

Collinear-Constrained-Attention

roformer

Rotary Transformer

Chinese-Vicuna

Chinese-Vicuna: A Chinese Instruction-following LLaMA-based Model —— 一个中文低资源的llama+lora方案，结构参考alpaca

lm-evaluation-harness

A framework for few-shot evaluation of language models.

exllama

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

baichuan-Dynamic-NTK-ALiBi

百川Dynamic NTK-ALiBi的代码实现：无需微调即可推理更长文本

LongBench

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

VisualGLM-6B

Chinese and English multimodal conversational language model | 多模态中英双语对话语言模型

CogView2

official code repo for paper "CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers"

CogView

Text-to-Image generation. The repo for NeurIPS 2021 paper "CogView: Mastering Text-to-Image Generation via Transformers".

CodeGeeX2

CodeGeeX2: A More Powerful Multilingual Code Generation Model

CodeGeeX

CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023)

ChatGLM3

ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型

reformer-pytorch

Reformer, the efficient Transformer, in Pytorch

yarn

YaRN: Efficient Context Window Extension of Large Language Models

Long-Context

This repository contains code and tooling for the Abacus.AI LLM Context Expansion project. Also included are evaluation scripts and benchmark tasks that evaluate a model’s information retrieval capabilities with context expansion. We also include key experimental results and instructions for reproducing and building on them.

CLEX

[ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models

LongLoRA

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

LLMLingua

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.