Kaiqiang Song's repositories
gpt-crawler
Crawl a site to generate knowledge files to create your own custom GPT from a URL
lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
flash-attention
Fast and memory-efficient exact attention
LLMDataHub
A quick guide (especially) for trending instruction finetuning datasets
awesome-RLHF
A curated list of reinforcement learning with human feedback resources (continually updated)
LLaMA-Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
transformers-bloom-inference
Fast Inference Solutions for BLOOM
helm
Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110).
ChatGPT
Lightweight package for interacting with ChatGPT's API by OpenAI. Uses reverse engineered official API.
DeepSpeech
DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.
multilingual-rouge
A multilingual rouge package (followed rouge_score) using BPE-tokenizer (from huggingface)
knn-transformers
PyTorch code for the RetoMaton paper: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an implementation of kNN-LM and kNN-MT
nlp-in-ling
Natural Language Processing Research in North American Linguistics Departments
AI-Paper-Collector
Fully-automated scripts for collecting AI-related papers
docAMR
code for document level AMR representation and evaluation
gitignore
A collection of useful .gitignore templates
nlp_chinese_corpus
大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP
DeepSpeedExamples
Example models using DeepSpeed
NLPDataSet
记录本人整理的一些数据集
DataLab
The unified platform for data-related resources.
summarization-datasets
Pre-processing and in some cases downloading of datasets for the paper "Content Selection in Deep Learning Models of Summarization."
DALLE2-pytorch
Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch
long-range-arena
Long Range Arena for Benchmarking Efficient Transformers
transformer-ls
Official implementation of Long-Short Transformer in PyTorch.