kingfan1998

rnn

一些RNN的实现

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

lamini

LOMO

LOMO: LOw-Memory Optimization

Train_Transformers_with_INT4

UltraChat

Large-scale, Informative, and Diverse Multi-round Chat Data (and Models)

Document-Plugin

Plug-and-Play Document Modules for Pre-trained Models

tree-of-thought-llm

[NeurIPS 2023] Tree of Thoughts: Deliberate Problem Solving with Large Language Models

qlora

QLoRA: Efficient Finetuning of Quantized LLMs

CSPostgraduate-408

💯 CSPostgraduate 计算机考研 408 专业课资料及真题资源

AlignScore

ACL2023 - AlignScore, a metric for factual consistency evaluation.

AlgoXY

Book of Elementary Functional Algorithms and Data structures

text-to-text-transfer-transformer

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

sentencepiece

Unsupervised text tokenizer for Neural Network-based text generation.

Distinct-N

Compute Distinct-N metric proposed by Jiwei Li et al.

Unilm

Luotuo-QA

骆驼QA，中文大语言阅读理解模型。

multilingual-t5

CPT

CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation

ChatGLM-6B

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

SeqDiffuSeq

Text Diffusion Model with Encoder-Decoder Transformers for Sequence-to-Sequence Generation [NAACL 2024]

DiffuSeq

Official Codebase for DiffuSeq

zero_nlp

中文nlp解决方案(大模型、数据、模型、训练、推理)

ChatGLM-Finetuning

基于ChatGLM-6B、ChatGLM2-6B、ChatGLM3-6B模型，进行下游具体任务微调，涉及Freeze、Lora、P-tuning、全参微调等

train-bert-pytorch

awesome-pretrained-chinese-nlp-models

Awesome Pretrained Chinese NLP Models，高质量中文预训练模型&大模型&多模态模型&大语言模型集合

MT5ForGeneration

基于精简mt5预训练模型的seq2seq结构的实现

MT5_chinese_simplify

pytorch版本MT5模型的中文精简代码

turkish-question-generation

Automated question generation and question answering from Turkish texts using text-to-text transformers

pytorch_med_T5-large_scale_pretraining_and_fientune-

基于T5 和 mt5 模型的医学nlp大规模预训练模型的训练和验证，测试