IST Austria Distributed Algorithms and Systems Lab

gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

sparsegpt

Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

marlin

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

qmoe

Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".

PanzaMail

QUIK

Repository for the QUIK project, enabling the use of 4bit kernels for generative inference

OBC

Code for the NeurIPS 2022 paper "Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning".

SparseFinetuning

Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry

RoSA

QIGen

Repository for CPU Kernel Generation for LLM Inference

Sparse-Marlin

spdy

Code for ICML 2022 paper "SPDY: Accurate Pruning with Speedup Guarantees"

sparseprop

peft-rosa

A fork of the PEFT library, supporting Robust Adaptation (RoSA)

CrAM

Code for reproducing the results from "CrAM: A Compression-Aware Minimizer" accepted at ICLR 2023

MicroAdam

This repository contains code for the MicroAdam paper.

spops

CAP

Repository for Correlation Aware Prune (NeurIPS23) source and experimental code

EFCP

The repository contains code to reproduce the experiments from our paper Error Feedback Can Accurately Compress Preconditioners available below:

ISTA-DASLab-Optimizers

pruned-vision-model-bias

Code for reproducing the paper "Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures"

TACO4NLP

Task aware compression for various NLP tasks

KDVR

Code for the experiments in Knowledge Distillation Performs Partial Variance Reduction, NeurIPS 2023

Mathador-LM

Code for the paper "Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on LLMs".

ZipLM

Code for the NeurIPS 2023 paper: "ZipLM: Inference-Aware Structured Pruning of Language Models".

AutoGPTQRoSA

FastOBQ-

GPTQ with finetuning

GridSearcher

GridSearcher simplifies running grid searches for machine learning projects in Python, emphasizing parallel execution and GPU scheduling without dependencies on SLURM or other workload managers.

llm-foundry

LLM training code for Databricks foundation models

SPADE

Code of SPADE: Sparsity Guided Debugging for Deep Neural Networks