Shiwei Liu's repositories
In-Time-Over-Parameterization
[ICML 2021] "Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training" by Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, Mykola Pechenizkiy
Awesome-Efficient-LLM
A curated list for Efficient Large Language Models
functorch
functorch is JAX-like composable function transforms for PyTorch.
LLM-QAT
Code repo for the paper "LLM-QAT Data-Free Quantization Aware Training for Large Language Models"
LLM-Shearing
Preprint: Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
metaseq
Repo for external large-scale work
OpenMoE
A family of open-sourced Mixture-of-Experts (MoE) Large Language Models
OWL
Official Pytorch Implementation of "Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity"
pytorch-image-models
PyTorch image models, scripts, pretrained weights -- ResNet, ResNeXT, EfficientNet, EfficientNetV2, NFNet, Vision Transformer, MixNet, MobileNet-V3/V2, RegNet, DPN, CSPNet, and more
shiweiliuiiiiiii.github.io
Github Pages template for academic personal websites, forked from mmistakes/minimal-mistakes
sparsegpt
Code for the paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".
SVAMP
NAACL 2021: Are NLP Models really able to Solve Simple Math Word Problems?
Versatile-Diffusion
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model, 2022
wanda
A simple and effective LLM pruning approach.
WeightWatcher
The WeightWatcher tool for predicting the accuracy of Deep Neural Networks