Hongsun_Jang's starred repositories
finetune-gpt2xl
Guide: Finetune GPT2-XL (1.5 Billion Parameters) and finetune GPT-NEO (2.7 B) on a single GPU with Huggingface Transformers using DeepSpeed
Pretrained-Language-Model
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
Ok-Topk
Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with the decentralized parallel Stochastic Gradient Descent (SGD) optimizer, and its convergence is proved theoretically and empirically.
computer-science
:mortar_board: Path to a free self-taught education in Computer Science!
SimpleSSD-FullSystem
Open-Source Licensed Educational SSD Simulator for High-Performance Storage and Full-System Evaluations
improved-diffusion
Release for Improved Denoising Diffusion Probabilistic Models
ChatGPT-as-a-server
Using ChatGPT as a real backend
lm-evaluation-harness
A framework for few-shot evaluation of language models.
CrystalDiskInfo
CrystalDiskInfo
ssd-benchmark-rs
Super Simple Disk Benchmark - benchmarks the writing performance of your disk
Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
MQSim
MQSim is a fast and accurate simulator modeling the performance of modern multi-queue (MQ) SSDs as well as traditional SATA based SSDs. MQSim faithfully models new high-bandwidth protocol implementations, steady-state SSD conditions, and the full end-to-end latency of requests in modern SSDs. It is described in detail in the FAST 2018 paper by Arash Tavakkol et al., "MQSim: A Framework for Enabling Realistic Studies of Modern Multi-Queue SSD Devices" (https://people.inf.ethz.ch/omutlu/pub/MQSim-SSD-simulation-framework_fast18.pdf)
Vitis-Tutorials
Vitis In-Depth Tutorials
Literatures-on-GNN-Acceleration
A reading list for deep graph learning acceleration.
GenStore
GenStore is the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. Described in the ASPLOS 2022 paper by Mansouri Ghiasi et al. at https://people.inf.ethz.ch/omutlu/pub/GenStore_asplos22-arxiv.pdf