optimizers

There are 8 repositories under optimizers topic.

annotated_deep_learning_paper_implementations
labmlai / annotated_deep_learning_paper_implementations
🧑‍🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
attention deep-learning deep-learning-tutorial gan literate-programming machine-learning neural-networks optimizers pytorch reinforcement-learning transformer transformers
Language:Python 52092
lucidrains / lion-pytorch
🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch
artificial-intelligence deep-learning evolutionary-search optimizers
Language:Python 1972
elixir-nx / axon
Nx-powered Neural Networks
deep-learning elixir neural-networks nx optimizers
Language:Elixir 1492
gugarosa / opytimizer
🐦 Opytimizer is a Python library consisting of meta-heuristic optimization algorithms.
artificial-intelligence bioinspired meta-heuristic meta-heuristic-optimization optimization optimizers python
Language:Python 597
Yonghongwei / Gradient-Centralization
A New Optimization Technique for Deep Neural Networks
deep-neural-networks gradient-centralization optimization-technique optimizers
Language:Python 531
CyberZHG / keras-radam
RAdam implemented in Keras & TensorFlow
adam keras optimizers radam rectified-adam tensorflow
Language:Python 324
OverLordGoldDragon / keras-adamw
Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers
keras optimizers adamw adamwr nadam sgd learning-rate-multipliers warm-restarts tensorflow
Language:Python 166
clovaai / frostnet
FrostNet: Towards Quantization-Aware Network Architecture Search
classification computer-vision deep-learning int8-quantization network-architecture object-detection optimizers post-quantization pytorch quantization quantization-aware-training quantization-efficient-network semantic-segmentation style-transfer
Language:Python 106
Gradient-Centralization-TensorFlow
Rishit-dagli / Gradient-Centralization-TensorFlow
Instantly improve your training performance of TensorFlow models with just 2 lines of code!
deep-learning deep-neural-networks gradient-centralization machine-learning neural-network optimization-technique optimizers pip python python-package python3 tensorflow tensorflow2
Language:Python 106
DL4S
palle-k / DL4S
Accelerated tensor operations and dynamic neural networks based on reverse mode automatic differentiation for every device that can run Swift - from watchOS to Linux
deep-learning deep-neural-networks gradient-descent automatic-differentiation autograd tensor recurrent-neural-networks convolutional-neural-networks machine-learning neural-networks swift swift-machine-learning recurrent-networks derivatives optimizers
Language:Swift 102
usef-kh / fer
Code for the paper "Facial Emotion Recognition: State of the Art Performance on FER2013"
saliency-maps optimizers architecture tuning loss vggnet fer2013 fer facial-emotions schedulers
Language:Jupyter Notebook 95
decile-team / submodlib
Summarize Massive Datasets using Submodular Optimization
greedy-algorithms combinatorial-optimization submodular-optimization data-summarization submodular-functions submodular submodular-information-measures optimizers
Language:Jupyter Notebook 83
hfxunlp / transformer
Neutron: A pytorch based implementation of Transformer and its variants.
neural-machine-translation transformer seq2seq natural-language-processing attention-is-all-you-need average-attention-network dynamic-sentence-sampling robust-neural-machine-translation beam-search ensemble average-models pytorch python3 multi-gpu sentential-context context-aware-nmt relative-position optimizers dynamic-batch-size multilingual-nmt
Language:Python 63
CyberZHG / keras-lookahead
Lookahead mechanism for optimizers in Keras.
keras optimizers lookahead
Language:Python 49
ivallesp / awesome-optimizers
Neural Network optimizers implemented from scratch in numpy (Adam, Adadelta, RMSProp, SGD, etc.)
deep-learning gradient-descent optimizers
Language:Jupyter Notebook 47
ProudElm / solaredgeoptimizers
Intergration to get optimizers information from the SolarEdge portal
optimizers python solaredge-api
Language:Python 43
warner-benjamin / optimi
Fast, Modern, Memory Efficient, and Low Precision PyTorch Optimizers
deep-learning optimizers pytorch
Language:Python 42
shreyansh26 / ML-Optimizers-JAX
Toy implementations of some popular ML optimizers using Python/JAX
jax optimization-algorithms optimizers machine-learning gradient-descent adam-optimizer adam momentum
Language:Python 41
ShivamShrirao / dnn_from_scratch
A high level deep learning library for Convolutional Neural Networks,GANs and more, made from scratch(numpy/cupy implementation).
convolutional-neural-networks backpropagate cnn gan generative-adversarial-network deep-learning mnist cifar numpy optimizers library python computer-vision cnn-from-scratch ml-from-scratch machine-learning neural-network deep-neural-networks keras
Language:Python 30
nestordemeure / flaxOptimizers
A collection of optimizers, some arcane others well known, for Flax.
flax optimizers
Language:Python 29
sail-sg / optim4rl
Optim4RL is a Jax framework of learning to optimize for reinforcement learning.
learning-to-learn learning-to-optimize meta-learning meta-learning-algorithm meta-learning-algorithms optimization optimization-algorithms optimizer optimizers reinforcement-learning reinforcement-learning-algorithms jax
Language:Python 21
devzhk / cgds-package
Package for CGD and ACGD optimizers
pytorch optimizers
Language:Python 19
harshalmittal4 / Hypergradient_variants
Improved Hypergradient optimizers, providing better generalization and faster convergence.
hypergradient optimizers learning-rate step-size adam-optimizer momentum
Language:Jupyter Notebook 16
chanzer0 / NBA-DFS-Tools
A set of NBA optimizers and GPP tools to help you win daily fantasy sports
draftkings ownership optimizers fanduel player-data
Language:Python 13
OrigamiDream / lion-tf
Lion - EvoLved Sign Momentum w/ New Optimizer API in TensorFlow 2.11+
deep-learning optimizers tensorflow
Language:Python 10
dmeoli / optiml
Optimizers for/and sklearn compatible Machine Learning models
machine-learning numerical-optimization optimization artificial-intelligence svm neural-network optimization-algorithms constrained-optimization unconstrained-optimization convex-optimization nonlinear-optimization quadratic-programming lagrangian-relaxation nonsmooth-optimization sklearn-compatible deep-neural-networks duality optimizers
Language:Python 8
seungjaeryanlee / awesome-optimizers
A curated list of optimizers for machine learning.
machine-learning optimizers tensorflow pytorch
8
dplassgit / d2lang
D2 is a strongly-typed, statically-typed, (mostly) inferred-type compiled language.
compiler language programming-language compiler-construction compiler-optimization optimizers
Language:Java 6
plusminuschirag / Optimizers-Visualizations
A Repository to Visualize the training of Linear Model by optimizers such as SGD, Adam, RMSProp, AdamW, ASMGrad etc
python3 tensorflow pytorch optimizers rmsprop matplotlib numpy sgd-optimizer adam-optimizer amsgrad
Language:Jupyter Notebook 5
timvvvht / Neural-Networks-and-Optimizers-from-scratch
Neural Networks and optimizers from scratch in NumPy, featuring newer optimizers such as DemonAdam or QHAdam.
neural-networks backpropagation neural-networks-from-scratch numpy mnist-dataset optimizers demon-adam nesterov-accelerated-sgd rmsprop adadelta qhadam
Language:Jupyter Notebook 5
AqwamCreates / DataPredict
Lua-Based Machine, Deep And Reinforcement Learning Library (For Roblox And Pure Lua). Contains 34 Models!
classification clustering deep-learning lua machine-learning regression reinforcement-learning roblox software-library supervised-learning unsupervised-learning deep-reinforcement-learning optimizers
Language:Lua 4
ZO-AdaMM-vs-FO-AdaMM-convergence-and-minima-shape-comparison
EliaFantini / ZO-AdaMM-vs-FO-AdaMM-convergence-and-minima-shape-comparison
Implementation and comparison of zero order vs first order method on the AdaMM (aka AMSGrad) optimizer: analysis of convergence rates and minima shape
amsgrad cnn-filters convergence-analysis convergence-rate cosine-similarity deep-learning first-order-methods machine-learning optimization-algorithms optimizers python pytorch t-sne zero-order-methods first-order-adamm minima-analysis zero-order-adammm zo-sgd
Language:Jupyter Notebook 4
Kockiee / Kockie-Optimizer-Python
Aplicação Windows desenvolvida por mim para otimização tanto do sistema como de certos jogos
desktop-application optimizers windows-10
Language:Python 4
Raffaelbdl / hyperbolic-nn-haiku
dm-haiku implementation of hyperbolic neural networks
dm-haiku hyperbolic-geometry jax neural-networks optax optimizers
Language:Python 4
dzonimn / Zoolia.jl
Julia implementations of various animal-inspired optimizers
grey-wolf-optimizer optimisation optimisation-algorithms optimization optimization-algorithms optimization-methods optimizer optimizers whale-optimization whale-optimization-algorithm
Language:Julia 3
tianshijing / Awesome-Optimizers
This repository is a meticulously assembled anthology of optimization algorithms, meticulously implemented in PyTorch, designed to cater to the diverse needs of the machine learning research community.
machine-learning optimizers deep-neural-networks optimization-algorithms
Language:Jupyter Notebook 3

optimizers

labmlai / annotated_deep_learning_paper_implementations

lucidrains / lion-pytorch

elixir-nx / axon

gugarosa / opytimizer

Yonghongwei / Gradient-Centralization

CyberZHG / keras-radam

OverLordGoldDragon / keras-adamw

clovaai / frostnet

Rishit-dagli / Gradient-Centralization-TensorFlow

palle-k / DL4S

usef-kh / fer

decile-team / submodlib

hfxunlp / transformer

CyberZHG / keras-lookahead

ivallesp / awesome-optimizers

ProudElm / solaredgeoptimizers

warner-benjamin / optimi

shreyansh26 / ML-Optimizers-JAX

ShivamShrirao / dnn_from_scratch

nestordemeure / flaxOptimizers

sail-sg / optim4rl

devzhk / cgds-package

harshalmittal4 / Hypergradient_variants

chanzer0 / NBA-DFS-Tools

OrigamiDream / lion-tf

dmeoli / optiml

seungjaeryanlee / awesome-optimizers

dplassgit / d2lang

plusminuschirag / Optimizers-Visualizations

timvvvht / Neural-Networks-and-Optimizers-from-scratch

AqwamCreates / DataPredict

EliaFantini / ZO-AdaMM-vs-FO-AdaMM-convergence-and-minima-shape-comparison

Kockiee / Kockie-Optimizer-Python

Raffaelbdl / hyperbolic-nn-haiku

dzonimn / Zoolia.jl

tianshijing / Awesome-Optimizers