sparse-neural-networks

There are 4 repositories under sparse-neural-networks topic.

dcmocanu / sparse-evolutionary-artificial-neural-networks
Always sparse. Never dense. But never say never. A Sparse Training repository for the Adaptive Sparse Connectivity concept and its algorithmic instantiation, i.e. Sparse Evolutionary Training, to boost Deep Learning scalability on various aspects (e.g. memory and computational time efficiency, representation and generalization power).
adaptive-sparse-connectivity artificial-neural-networks classification complex-networks deep-learning deep-learning-algorithms deep-learning-papers deep-neural-networks evolutionary-algorithms generative-models multi-layer-perceptron neuroevolution randomization restricted-boltzmann-machine scalability scalable-deep-learning sparse-evolutionary-training sparse-neural-networks sparse-training sparsity
Language:Python 250
nanowell / Q-Sparse-LLM
My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
artificial-intelligence deep-neural-networks deeplearning large-language-models machine-learning q-sparse sparse-neural-networks transformer
Language:Python 33
VITA-Group / SFW-Once-for-All-Pruning
[ICLR 2022] "Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, and No Retraining" by Lu Miao*, Xiaolong Luo*, Tianlong Chen, Wuyang Chen, Dong Liu, Zhangyang Wang
pruning stochastic-frank-wolfe once-and-for-all sparse-neural-networks
Language:Python 32
VITA-Group / SMC-Bench
[ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen, Tianjin Huang, AJAY KUMAR JAISWAL, Zhangyang Wang
benchmark deep-learning dynamic-sparse-training pruning sparse-neural-networks sparsity
Language:Python 28
Efficient-Scalable-Machine-Learning / EvNN
Event-based neural networks
cuda efficient-neural-networks neural-network pytorch rnn-model sparse-neural-networks spiking-neural-networks
Language:Python 27
zahraatashgahi / QuickSelection
[Machine Learning Journal (ECML-PKDD 2022 journal track)] Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders
feature-selection autoencoder denoising-autoencoders sparse-neural-networks sparsity dimensionality-reduction deep-learning sparse-training
Language:Python 17
lim142857 / Sparsifiner
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
efficient-transformers efficient-vision-transformers fast-inference sparse-attention attention-mechanism low-rank sparse-neural-networks vision-transformer
Language:Python 15
GhadaSokar / Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning
[IJCAI 2022] "Dynamic Sparse Training for Deep Reinforcement Learning" by Ghada Sokar, Elena Mocanu , Decebal Constantin Mocanu, Mykola Pechenizkiy, and Peter Stone.
sparse-neural-networks sparsity reinforcement-learning deep-reinforcement-learning continuous-control-tasks sparse-training deep-learning deep-neural-networks
Language:Python 14
IlanPrice / DCTpS
Code for testing DCT plus Sparse (DCTpS) networks
deep-learning sparse-neural-networks neural-network-pruning neural-network-compression
Language:Python 14
bcrafton / ssdfa
Github page for SSDFA
bio-plausible-algorithms feedback-alignment local-learning backpropagation sparse-neural-networks hardware-acceleration
Language:Python 11
cambridge-mlg / arch_uncert
Code for "Variational Depth Search in ResNets" (https://arxiv.org/abs/2002.02797)
bayesian-inference expectation-maximization neural-architecture-search bayesian-neural-networks bayesian-deep-learning variational-inference reproducible-research reproducible-paper learnt-depth residual-networks resnets network-depths mnist fashion-mnist svhn uncertainty sparse-neural-networks network-compression
Language:Jupyter Notebook 9
GhadaSokar / SpaceNet
Implementation for the paper "SpaceNet: Make Free Space For Continual Learning" in PyTorch.
sparse-representations sparse-neural-networks sparse-training continual-learning life-long-learning incremental-learning catastrophic-forgetting brain-inspired deep-learning deep-neural-networks
Language:Python 9
VITA-Group / Peek-a-Boo
[ICLR 2022] "Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently", by Xiaohan Chen, Jason Zhang and Zhangyang Wang.
deep-learning sparse-neural-networks efficient-deep-learning
Language:Python 7
zahraatashgahi / NeuroFS
[TMLR] Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks
deep-learning dynamic-sparse-training efficiency feature-selection high-dimensional-data keras neural-networks python sparse-neural-networks sparse-training
Language:Python 7
A-Klass / torch_topkast
PyTorch Implementation of TopKAST
deep-learning deep-neural-networks sparse-neural-networks sparse-training sparsity
Language:Python 6
BICLab / RM-SNN
Offical implementation of "Sparser spiking activity can be better: Feature Refine-and-Mask spiking neural network for event-based visual recognition" (Neural Networks 2023)
attention-mechanism sparse-neural-networks spiking-neural-networks
Language:Python 5
SurrealVectors / Soevnn
A neural net with a terminal-based testing program.
neural-networks neural-network machine-learning sparse-neural-networks recurrent-neural-networks
Language:F# 4
zahraatashgahi / CTRE
[Machine Learning Journal (ECML-PKDD 2022 journal track)] A Brain-inspired Algorithm for Training Highly Sparse Neural Networks
sparse-neural-networks sparsity sparse-training machine-learning deep-learning mlp classification
Language:Python 4
BrosnanYuen / RayBNN_Neural
Neural Networks with Sparse Weights in Rust using GPUs, CPUs, and FPGAs via CUDA, OpenCL, and oneAPI
cpu cuda deep-learning gpu machine-learning machine-learning-algorithms neural-network neural-networks opencl raybnn rust sparse-network sparse-neural-networks parallel
Language:Rust 3
elch10 / SignalNeuroHackTasks
Implementation of artcile "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks"
sparse-neural-networks conv-lenet-5 lenet-300-100 pytorch-implementation
Language:Jupyter Notebook 3
zahraatashgahi / Neuron-Attribution
[ECAI 2024] Unveiling the Power of Sparse Neural Networks for Feature Selection
ecai ecai2024 feature-importance feature-selection sparse sparse-neural-networks sparsity neuron-attribution
Language:Python 2
zahraatashgahi / PALS
[ECML-PKDD 2024] Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers
efficiency forecasting pruning python pytorch sparse-neural-networks time-series-analysis transformers
Language:Python 2
ZIYU-DEEP / Generalization-and-Memorization-in-Sparse-Training
This is the repository for the SNN-22 Workshop paper on "Generalization and Memorization in Sparse Neural Networks".
deep-learning fisher-information-matrix sparse-neural-networks sparse-training
Language:Python 2
amikom-gace-research-group / characterize-pruning
Characterization study repository for pruning, a popular way to compress a DL model. this repo also investigates optimal sparse tensor layouts for pruned nets
characterization-study edge-devices model-compression pruning sparse-neural-networks
Language:Python 1
BrosnanYuen / RayBNN_Sparse
Sparse Matrix Library for GPUs, CPUs, and FPGAs via CUDA, OpenCL, and oneAPI
cpu cuda opencl parallel parallel-computing parallel-programming sparse sparse-coding sparse-matrix sparse-neural-networks arrayfire rust gpu gpu-computing raybnn
Language:Rust 1
Efficient-Scalable-Machine-Learning / DeepRewire
Easily create and optimize PyTorch networks as in the Deep Rewiring paper (https://igi-web.tugraz.at/PDF/241.pdf). Install using 'pip install deep_rewire'
deep-rewiring sparse sparse-neural-networks
Language:Python 1
guyez / Simple-Sparsely-Connected-NN
Simple C++ implementation of a sparsely connected multi-layer neural network using OpenMP and CUDA for parallelization.
cplusplus cpp neural-network sparse-neural-networks openmp openmp-parallelization openmp-optimization cuda cuda-programming cuda-kernels
Language:C++ 1
MarwanNour / Sparse-Neural-Networks-GPU
GPU Computing course project
gpu-acceleration sparse-neural-networks
Language:Cuda 1
DanielEftekhari / neural-network-pruning
Neural Network Sparsification via Pruning
pytorch deep-learning sparse-neural-networks pruning
Language:Python 0
neilkichler / robustness_set
Robustness of Sparse Multilayer Perceptrons for Supervised Feature Selection
feature-selection multilayer-perceptron robustness sparse-neural-networks
Language:Python 0
zitonghuangcynthia / Simultaneous-Weight-and-Architecture-Optimization
[NeurIPS 2024 FITML Workshop] This is the official code for the paper "Simultaneous Weight and Architecture Optimization for Neural Networks"
autoencoder deep-learning neural-architecture-search sparse-neural-networks neural-network-search
Language:Python 0
mxmnburger / structure-of-lottery-tickets
Master's Thesis Project - Lottery Tickets contain independent subnetworks when trained on independent tasks.
independence lottery-ticket-hypothesis sparse-neural-networks
Language:Jupyter Notebook

sparse-neural-networks

dcmocanu / sparse-evolutionary-artificial-neural-networks

nanowell / Q-Sparse-LLM

VITA-Group / SFW-Once-for-All-Pruning

VITA-Group / SMC-Bench

Efficient-Scalable-Machine-Learning / EvNN

zahraatashgahi / QuickSelection

lim142857 / Sparsifiner

GhadaSokar / Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning

IlanPrice / DCTpS

bcrafton / ssdfa

cambridge-mlg / arch_uncert

GhadaSokar / SpaceNet

VITA-Group / Peek-a-Boo

zahraatashgahi / NeuroFS

A-Klass / torch_topkast

BICLab / RM-SNN

SurrealVectors / Soevnn

zahraatashgahi / CTRE

BrosnanYuen / RayBNN_Neural

elch10 / SignalNeuroHackTasks

zahraatashgahi / Neuron-Attribution

zahraatashgahi / PALS

ZIYU-DEEP / Generalization-and-Memorization-in-Sparse-Training

amikom-gace-research-group / characterize-pruning

BrosnanYuen / RayBNN_Sparse

Efficient-Scalable-Machine-Learning / DeepRewire

guyez / Simple-Sparsely-Connected-NN

MarwanNour / Sparse-Neural-Networks-GPU

DanielEftekhari / neural-network-pruning

neilkichler / robustness_set

zitonghuangcynthia / Simultaneous-Weight-and-Architecture-Optimization

mxmnburger / structure-of-lottery-tickets