ckswjd99

Awesome-Quantization-Papers

List of papers related to neural network quantization in recent AI conferences and journals.

BiBench

[ICML 2023] This project is the official implementation of our accepted ICML 2023 paper BiBench: Benchmarking and Analyzing Network Binarization.

BinaryConnect

Implementation of BinaryConnect on Pytorch

XNNPACK

High-efficiency floating-point neural network inference operators for mobile, server, and Web

AdderNet

Code for paper " AdderNet: Do We Really Need Multiplications in Deep Learning?"

awesome-model-quantization

A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.

matmulfreellm

Implementation for MatMul-free LM.

Neural_Tangent_Kernel

cifar10-resnet

PyTorch implementation of a 9-layer ResNet for CIFAR-10.

pytorch-cifar

95.47% on CIFAR10 with PyTorch

pytorch-seq2seq

An open source framework for seq2seq models in PyTorch.

pytorch_resnet_cifar10

Proper implementation of ResNet-s for CIFAR10/100 in pytorch that matches description of the original paper.

MLP-Mixer-CIFAR

PyTorch implementation of Mixer-nano (#parameters is 0.67M, originally Mixer-S/16 has 18M) with 90.83 % acc. on CIFAR-10. Training from scratch.

RNN-on-wikitext2

Pytorch of RNN and LSTM on wikitext2 dataset

fwdgrad

Implementation of "Gradients without backpropagation" paper (https://arxiv.org/abs/2202.08587) using functorch

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

vision-transformers-cifar10

Let's train vision transformers (ViT) for cifar 10!

gpt-2-Pytorch

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

DeepZero

[ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Diffenderfer, Jiancheng Liu, Konstantinos Parasyris, Yihua Zhang, Zheng Zhang, Bhavya Kailkhura, Sijia Liu

feedback-alignment-pytorch

Implementation of feedback alignment learning in PyTorch

MeZO

[NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333

pytorch-cifar100

Practice on cifar100(ResNet, DenseNet, VGG, GoogleNet, InceptionV3, InceptionV4, Inception-ResNetv2, Xception, Resnet In Resnet, ResNext,ShuffleNet, ShuffleNetv2, MobileNet, MobileNetv2, SqueezeNet, NasNet, Residual Attention Network, SENet, WideResNet)

Knowledge-Inheritance

Source code for paper: Knowledge Inheritance for Pre-trained Language Models

LiGO

[ICLR 2023] "Learning to Grow Pretrained Models for Efficient Transformer Training" by Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen, Philip Greengard, Leonid Karlinsky, Rogerio Feris, David Cox, Zhangyang Wang, Yoon Kim

growneuron

clustered-imagenet-labels

A dataset for production-friendly image classifiers

vit-pytorch

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

pytorch-vgg-cifar10

This is the PyTorch implementation of VGG network trained on CIFAR10 dataset

nccl

Optimized primitives for collective multi-GPU communication

ASPEN

This is the proof-of-concept CPU implementation of ASPEN used for the NeurIPS'23 paper ASPEN: Breaking Operator Barriers for Efficient Parallelization of Deep Neural Networks.