tunsheng

TorchLitho

Differentiable Computational Lithogrpahy Framework

rigl

End-to-end training of sparse deep neural networks with little-to-no performance loss.

Tutorial-SCADS-Summer-School-2020-Scalable-Deep-Learning

Code associated with 6th International (online) Summer school on AI and Big Data tutorial "Scalable Deep Learning Tutorial" and "Scalable deep learning: how far is one billion neurons?" tutorial at ECAI 2020.

SoftAdapt

Implementation of the SoftAdapt paper (techniques for adaptive loss balancing of multi-tasking neural networks)

SMDP

Solving Inverse Physics Problems with Score Matching

RoBO

RoBO: a Robust Bayesian Optimization framework

Pruning-Weights-with-Biobjective-Optimization-Keras

Overparameterization and overfitting are common concerns when designing and training deep neural networks. Network pruning is an effective strategy used to reduce or limit the network complexity, but often suffers from time and computational intensive procedures to identify the most important connections and best performing hyperparameters. We suggest a pruning strategy which is completely integrated in the training process and which requires only marginal extra computational cost. The method relies on unstructured weight pruning which is re-interpreted in a multiobjective learning approach. A batchwise Pruning strategy is selected to be compared using different optimization methods, of which one is a multiobjective optimization algorithm. As it takes over the choice of the weighting of the objective functions, it has a great advantage in terms of reducing the time consuming hyperparameter search each neural network training suffers from. Without any a priori training, post training, or parameter fine tuning we achieve highly reductions of the dense layers of two commonly used convolution neural networks (CNNs) resulting in only a marginal loss of performance. Our results empirically demonstrate that dense layers are overparameterized as with reducing up to 98 % of its edges they provide almost the same results. We contradict the theory that retraining after pruning neural networks is of great importance and opens new insights into the usage of multiobjective optimization techniques in machine learning algorithms in a Keras framework. The Stochastic Multi Gradient Descent Algorithm implementation in Python3 is for usage with Keras and adopted from paper of S. Liu and L. N. Vicente: "The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning". It is combined with weight pruning strategies to reduce network complexity and inference time.

fastfeedforward

A repository for log-time feedforward networks

MacroMax

Library for solving the macroscopic Maxwell equations in complex dielectric materials. The materials may be any mixture of isotropic and anisotropic permittivity, permeability, and coupling tensors.

MultirateTrainingOfNNs

Supplement code to our ICML 2022 paper on Multirate Training of Neural Networks

autobound

AutoBound automatically computes upper and lower bounds on functions.

Koopman-Training-Pytorch-Tools

Tools to perform Koopman training in Pytorch

allegro

Allegro is an open-source code for building highly scalable and accurate equivariant deep learning interatomic potentials

nif

A library for dimensionality reduction on spatial-temporal PDE

composer

Supercharge Your Model Training

koopman-forecasting

Long-term probabilistic forecasting of quasiperiodic phenomena using Koopman theory

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

deep-symbolic-optimization

A deep learning framework for symbolic optimization.

STIMD

py-metal-compute

A python library to run metal compute kernels on macOS

sepsis_competition_physionet_2019

Code (rewritten) for our winning submission to the sepsis physionet 2019 challenge. Team name: Can I get your signature?

referseg_rrn

omnipose

Omnipose: a high-precision solution for morphology-independent cell segmentation

leaf-audio

LEAF is a learnable alternative to audio features such as mel-filterbanks, that can be initialized as an approximation of mel-filterbanks, and then be trained for the task at hand, while using a very small number of parameters.

leaf-audio-pytorch

Pytorch port of Google Research's LEAF Audio paper

leaf-pytorch

PyTorch implementation of the LEAF audio frontend

Plot2Spec

an automatic plot digitizer for spectroscopy images (i.e. XANES and Raman)

copy-paste-aug

Copy-paste augmentation for segmentation and detection tasks

LiteMORT

A memory efficient GBDT on adaptive distributions. Much faster than LightGBM with higher accuracy. Implicit merge operation.

CDGP

Counterexample-Driven Genetic Programming