luuyin

EdgeDeviceLLMCompetition-Starting-Kit

welore-public

OwLore

Official Pytorch Implementation of "OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning" by Pengxiang Li, Lu Yin, Xiaowei Gao, Shiwei Liu

Junk_DNA_Hypothesis

"Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity" Lu Yin, Shiwei Liu, Ajay Jaiswal, Souvik Kundu, Zhangyang Wang

chase

Dynamic Sparsity Is Channel-Level Sparsity Learner [Neurips 2023]

OWL

Official Pytorch Implementation of "Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity"

ILM-VP

[CVPR23] "Understanding and Improving Visual Prompting: A Label-Mapping Perspective" by Aochuan Chen, Yuguang Yao, Pin-Yu Chen, Yihua Zhang, and Sijia Liu

DSify

Boosting Driving Scene Understanding with Advanced Vision-Language Models

BERT-Tickets

[NeurIPS 2020] "The Lottery Ticket Hypothesis for Pre-trained BERT Networks", Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, Michael Carbin

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Generating-the-simple-shape-dataset

Generate a simple shape dataset with different colors, shapes, thicknesses, and heights.

UGTs-LoG

This is the official code for UGTs.

mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

mixture-of-experts

PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

git-re-basin-pytorch

Git Re-Basin: Merging Models modulo Permutation Symmetries in PyTorch

Sup-tickets

Lottery-pools

DynamicReLU

Implementation of Dynamic ReLU on Pytorch

SLaK

[ICLR 2023] "More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity"; [ICML 2023] "Are Large Kernels Better Teachers than Transformers for ConvNets?"

html-resume

A single-page resume template completely typeset with HTML & CSS.

al-folio

A beautiful, simple, clean, and responsive Jekyll theme for academics

Net2Net

Net2Net implementation on PyTorch for any possible vision layers.

Firefly

Offical Repo for Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks. Accepted by Neurips 2020.

openrazer

Open source driver and user-space daemon to control Razer lighting and other features on GNU/Linux

DCTpS

Code for testing DCT plus Sparse (DCTpS) networks

Learning-Loss-for-Active-Learning

Reproducing experimental results of LL4AL [Yoo et al. 2019 CVPR]

SET-MLP-ONE-MILLION-NEURONS

[Neural Computing and Applications] "Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware" by Shiwei Liu, Decebal Constantin Mocanu, Mykola Pechenizkiy

Random_Pruning

[ICLR 2022] The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training by Shiwei Liu, Tianlong Chen, Xiaohan Chen, Li Shen, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy

Selfish-RNN

[ICML 2021] "Selfish Sparse RNN Training" by Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei, Mykola Pechenizkiy

GraNet

[Neurips 2021] Sparse Training via Boosting Pruning Plasticity with Neuroregeneration