second-order-optimization

There are 2 repositories under second-order-optimization topic.

amirgholami / PyHessian
PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks
hessian second-order second-order-optimization hessian-free pytorch-library
Language:Jupyter Notebook 756
amirgholami / adahessian
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
adahessian hessian hessian-free optimizer second-order-optimization
Language:Python 279
lixilinx / psgd_torch
Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation preconditioner and more)
lie-groups pytorch second-order-optimization hessian-vector-product deep-learning optimization-algorithms preconditioner stochastic-gradient-descent affine-group low-rank-approximation kronecker-factored-approximation
Language:Python 171
Epigraph
EmbersArc / Epigraph
A C++ interface to formulate and solve linear, quadratic and second order cone problems.
numerical-optimization socp second-order-optimization quadratic-programming eigen-library qp eigen
Language:C++ 155
kron_torch
evanatyourservice / kron_torch
An implementation of PSGD Kron second-order optimizer for PyTorch
machine-learning optimization pytorch second-order-optimization
Language:Python 86
gpauloski / kfac-pytorch
Distributed K-FAC Preconditioner for PyTorch
pytorch optimizer kfac distributed second-order-optimization deep-learning
Language:Python 85
CharlieDinh / FEDL
FEDL-Federated Learning algorithm using TensorFlow (Transaction on Networking 2021)
federated-learning fedl second-order-optimization dane newton
Language:Python 57
CharlieDinh / FEDL_pytorch
This repository implements FEDL using pytorch
pytorch federated-learning second-order-optimization fedl
Language:Python 54
ltatzel / PyTorchHessianFree
PyTorch implementation of the Hessian-free optimizer
conjugate-gradient deep-learning deep-neural-networks ggn hessian-free matrix-free newtons-method second-order-optimization pytorch-implementation optimization-algorithms stochastic-optimizers
Language:Python 36
lixilinx / psgd_tf
Tensorflow implementation of preconditioned stochastic gradient descent
deep-learning hessian-vector-product lie-groups optimization-algorithms preconditioner second-order-optimization stochastic-gradient-descent tensorflow affine-group low-rank-approximation
Language:Python 34
psgd_jax
evanatyourservice / psgd_jax
Implementation of PSGD optimizer in JAX
jax optax optimization preconditioning psgd second-order-optimization
Language:Python 30
tomoleary / hessianlearn
Hessian-based stochastic optimization in TensorFlow and keras
optimization optimization-algorithms stochastic-optimization newton-method second-order-optimization tensorflow keras
Language:Python 28
kenshi84 / compatible-intrinsic-triangulations
Compatible Intrinsic Triangulations (SIGGRAPH 2022)
bijection bijective-maps computer-graphics cross-parameterization geometry-processing inter-surface-mapping parameterization second-order-optimization triangle-mesh compatible-triangulation intrinsic-triangulation siggraph siggraph2022
Language:C++ 27
OPTAMI / OPTAMI
This package is dedicated to high-order optimization methods. All the methods can be used similarly to standard PyTorch optimizers.
high-order-methods second-order-optimization pytorch-optimizers damped-netwon newton-method
Language:Python 25
renyiryry / kbfgs_neurips2020_public
second-order-optimization optimizer pytorch
Language:Jupyter Notebook 21
CharlieDinh / DONE
Federated Learning using PyTorch. Second-Order for Federated Learning. (IEEE Transactions on Parallel and Distributed Systems 2022)
federated-edge-learning second-order-optimization done federated-learning pytorch
Language:Jupyter Notebook 18
AntoinePassemiers / Beyond-Gradient-Descent
Minimalist deep learning library with first and second-order optimization algorithms made for educational purpose
neural-networks backpropagation-algorithm second-order-optimization gradient-descent
Language:Python 11
jmdvinodjmd / LIBS2ML
LIBS2ML: A Library for Scalable Second Order Machine Learning Algorithms
large-scale-learning stochastic stochastic-optimization c-plus-plus mex-files matlab matlab-script octave-scripts big-data second-order-optimization machine-learning-algorithms
Language:C++ 10
hiroyuki-kasai / Subsampled-RTR
Subsampled Riemannian trust-region (RTR) algorithms
riemannian-optimization trust-region stochastic-optimizers riemannian-manifold big-data second-order-optimization subsample sub-sampling sampling-methods riemannian-geometry gradient-descent-algorithm optimization optimization-algorithms stochastic-gradient-descent large-scale-optimizations
Language:MATLAB 7
tomoleary / MatrixFreeNewton.jl
Prototyping of matrix free Newton methods in Julia
julia optimization newton-method second-order-optimization optimization-algorithms stochastic-optimization
Language:Julia 7
gmatilde / SGN
An efficient and easy-to-use Theano implementation of the stochastic Gauss-Newton method for training deep neural networks.
neural-networks optimization optimization-algorithms convolutional-neural-networks numerical-methods stochastic-gradient-descent stochastic-optimization second-order-optimization gauss-newton-method
Language:Jupyter Notebook 6
hsivan / fosi
FOSI library for improving first order optimizers with second order information
jax optimizer pytorch second-order-optimization hessian-vector-product deep-learning lanczos
Language:Python 6
newoptpy
dahyun-kang / newoptpy
Newton’s second-order optimization methods in python
newton-method newtons-method optimization-algorithms quasi-newton quasi-newton-method second-order-optimization
Language:Python 4
IST-DASLab / EFCP
The repository contains code to reproduce the experiments from our paper Error Feedback Can Accurately Compress Preconditioners available below:
adaptive-gradient-method computer-vision deep-learning ggt language-model low-rank-approximation second-order-optimization top-k full-matrix-preconditioning m-fac
Language:Python 4
qiuweili / altmin
Second-Order Convergence of Alternating Minimizations
alternating-minimization nonconvex second-order-optimization global-optimization low-rank-factorization low-rank-matrix-recovery matrix-completion alternating-optimization proximal-algorithms matrix-factorization
4
renyiryry / tnt_neurips_2021
pytorch deep-learning optimizer second-order-optimization
Language:Jupyter Notebook 4
yorkerlin / StructuredNGD-DL
Matrix-multiplication-only KFAC; Code for ICML 2023 paper on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
kfac natural-gradients optimization-methods optimizer pytorch-implementation second-order-optimization fisher deep-learning hessian-free
Language:Python 4
cor3bit / awesome-soms
A curated list of resources for second-order stochastic optimization
awesome machine-learning optimization second-order-optimization jax
3
cor3bit / somax
Stochastic Second-Order Methods in JAX
jax machine-learning second-order-optimization gauss-newton-method hessian-free natural-gradient quasi-newton stochastic-optimization
Language:Python 3
inikishev / torchzero
Modular optimization library for PyTorch. I changed the API a lot and that will soon be pushed here💀
derivative-free-optimization gradient-descent gradient-free-optimization newton-raphson optimization optimization-algorithms optimizers pytorch second-order-optimization zeroth-order-optimization first-order-optimization gradient-approximations grafting
Language:Python 3
yangorwell / NGPlus
NG+: A new second-order optimizer for deep learning
second-order-optimization optimization
Language:Python 3
evanatyourservice / flat-sophia
sophia optimizer further projected towards flat areas of loss landscape
jax optax optimization second-order-optimization sophia
Language:Python 2
evanatyourservice / ordax
A collection of second-order optimizers and experiments in JAX
jax optimization second-order-optimization
2
sayarghoshroy / Optimization_and_Learning
Concepts and algorithms in core learning theory
optimization-algorithms gradient-descent regression classification second-order-optimization python
Language:Jupyter Notebook 2
aniruddhavpatil / second-order-model-selection
Regularization, Bayesian Model Selection and k-fold Cross-Validation Selection
bayesian-model-selection cross-validation-selection second-order-optimization machine-le
Language:Python 1
adahessian
riccardocadei / adahessian
Discussion of advantages and disadvantages of AdaHessian, a state-of-the-art Second Order Methods over First Order Methods on a Non-Convex Optimization Problem (digits classification on MNIST database using ResNet18). - @ EPFL
non-convex-optimization deep-learning second-order-optimization adahessian mnist
Language:Jupyter Notebook 1

second-order-optimization

amirgholami / PyHessian

amirgholami / adahessian

lixilinx / psgd_torch

EmbersArc / Epigraph

evanatyourservice / kron_torch

gpauloski / kfac-pytorch

CharlieDinh / FEDL

CharlieDinh / FEDL_pytorch

ltatzel / PyTorchHessianFree

lixilinx / psgd_tf

evanatyourservice / psgd_jax

tomoleary / hessianlearn

kenshi84 / compatible-intrinsic-triangulations

OPTAMI / OPTAMI

renyiryry / kbfgs_neurips2020_public

CharlieDinh / DONE

AntoinePassemiers / Beyond-Gradient-Descent

jmdvinodjmd / LIBS2ML

hiroyuki-kasai / Subsampled-RTR

tomoleary / MatrixFreeNewton.jl

gmatilde / SGN

hsivan / fosi

dahyun-kang / newoptpy

IST-DASLab / EFCP

qiuweili / altmin

renyiryry / tnt_neurips_2021

yorkerlin / StructuredNGD-DL

cor3bit / awesome-soms

cor3bit / somax

inikishev / torchzero

yangorwell / NGPlus

evanatyourservice / flat-sophia

evanatyourservice / ordax

sayarghoshroy / Optimization_and_Learning

aniruddhavpatil / second-order-model-selection

riccardocadei / adahessian