Doohae Jung (wavy.ocean)'s repositories
DenseRetrieval
Implementation of DPR, GradCache, DSI etc.
accelerate
🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
DeepSpeedExamples
Example models using DeepSpeed
GradCache
Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
Megatron-LM
Ongoing research training transformer models at scale
RETRO-pytorch
Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
Building-Python-Web-APIs-with-FastAPI
Building Python Web APIs with FastAPI, published by Packt
Chatbot_data
Chatbot_data_for_Korean
course
The Hugging Face course
fairseq
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
FiD
Fusion-in-Decoder
kolmev
Evaluation for korean language models (e.g. bert, roberta, bart, t5, gpt2...)
lassl
Easy Language Model Pretraining leveraging Huggingface's Transformers and Datasets
llmss
LLM simple serving (tensor model parallel, pubsub, grpc)
lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
marcopolo
Marco Emilio Polo (/ˈmɑːrkoʊ ˈpoʊloʊ/ (listen), Venetian: [ˈmaɾko ˈpolo], Italian: [ˈmarko ˈpɔːlo] (listen); c. 1254 – January 8, 1324)[1] was a Venetian merchant,[2][3] explorer, and writer from the Republic of Venice who travelled through Asia along the Silk Road between 1271 and 1295.
Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
tppys
Text processing by pyspark (just sample project)
tpu-starter
Everything you want to know about Google Cloud TPU
transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
trlx
A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)
YaLM-100B
Pretrained language model with 100B parameters