adam-optimizer

There are 2 repositories under adam-optimizer topic.

LiyuanLucasLiu / RAdam
On the Variance of the Adaptive Learning Rate and Beyond
optimizer adam adam-optimizer warmup
Language:Python 2549
parasdahal / deepnet
Deep learning library in plain Numpy.
cnn dropout batch-normalization adagrad adam-optimizer nesterov-accelerated-sgd
Language:Python 322
SirRob1997 / Crowded-Valley---Results
This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"
adam-optimizer deep-learning deep-learning-optimizers deep-neural-networks deeplearning sgd
184
tirtharajdash / CS-F425_Deep-Learning
CS F425 Deep Learning course at BITS Pilani (Goa Campus)
machine-learning deep-neural-networks bits-pilani generative-model optimisation adam-optimizer deep-learning
Language:Jupyter Notebook 111
YanaiEliyahu / AdasOptimizer
ADAS is short for Adaptive Step Size, it's an optimizer that unlike other optimizers that just normalize the derivative, it fine-tunes the step size, truly making step size scheduling obsolete, achieving state-of-the-art training performance
optimizer adam adam-optimizer artifical-neural-network deep-learning mnist machine-learning
Language:C++ 85
nengwp / Lion-vs-Adam
Lion and Adam optimization comparison
adam adam-optimizer lion lion-optimization vis
Language:Jupyter Notebook 64
aditya9211 / SVHN-CNN
Google Street View House Number(SVHN) Dataset, and classifying them through CNN
adam-optimizer cnn computer-vision confusion-matrix convolutional-neural-network convolutional-neural-networks dropout google-street grayscale-images house-numbers machine-learning mini-batch-gradient-descent mnist one-hot-encode svhn svhn-classifier tensorboard-visualizations tesnorflow
Language:Jupyter Notebook 63
v-iashin / CS231n
PyTorch/Tensorflow solutions for Stanford's CS231n: "CNNs for Visual Recognition"
computer-vision convolutional-neural-networks neural-network machine-learning stanford-university backpropagation recurrent-neural-networks lstm style-transfer generative-adversarial-network jupyter-notebook tensorflow svm-classifier adam-optimizer sgd-optimizer numpy solutions assignment pytorch deep-learning
Language:Jupyter Notebook 51
yashkant / padam-tensorflow
Reproducing the paper "PADAM: Closing The Generalization Gap of Adaptive Gradient Methods In Training Deep Neural Networks" for the ICLR 2019 Reproducibility Challenge
optimization tensorflow tensorflow-eager keras sgd-momentum adam-optimizer amsgrad padam wide-residual-networks
Language:Python 51
sagarvegad / Adam-optimizer
Implemented Adam optimizer in python
adam-optimizer gradient
Language:Python 48
shreyansh26 / ML-Optimizers-JAX
Toy implementations of some popular ML optimizers using Python/JAX
jax optimization-algorithms optimizers machine-learning gradient-descent adam-optimizer adam momentum
Language:Python 44
Arko98 / Gradient-Descent-Algorithms
A collection of various gradient descent algorithms implemented in Python from scratch
ai ml machine-learning machine-learning-algorithms deep-learning deep-learning-algorithms gradient-descent gradient-descent-algorithm rmsprop adagrad adam-optimizer nesterov-accelerated-sgd nesterov-momentum nesterov momentum-optimization-algorithm
Language:Python 39
VivianoRiccardo / Learning-Lab-C-Library
This library provides a set of functionalities for different type of deep learning (and ML) algorithms in C
deeplearning neural-network c library backpropagation softmax convolutional-layers residual-networks residual-layers nesterov adam-algorithm adam-optimizer avarage opencl optimization-algorithms transformer dropout-layers deep-learning lstm
Language:C 34
aromanro / MachineLearning
From linear regression towards neural networks...
adagrad adam-optimizer backpropagation data-analysis data-science generalized-linear-models gradient-descent linear-regression logistic-regression machine-learning machine-learning-algorithms multilayer-perceptron-network nadam nesterov-accelerated-sgd nesterov-momentum neural-network rmsprop
Language:C++ 27
rdspring1 / Count-Sketch-Optimizers
A compressed adaptive optimizer for training large-scale deep learning models using PyTorch
adam-optimizer sgd-optimizer deep-learning neural-network language-model count-sketch count-min-sketch sgd-momentum hashing adagrad imagenet transformer pytorch
Language:Python 27
RudreshVeerkhare / CustomXGBoost
Modified XGBoost implementation from scratch with Numpy using Adam and RSMProp optimizers.
adam-optimizer gradient-boosting-machine implementation-of-algorithms numpy rmsprop xgboost-algorithm
Language:Jupyter Notebook 26
yaricom / TimeSeriesLearning
The project aimed to implement Deep NN / RNN based solution in order to develop flexible methods that are able to adaptively fillin, backfill, and predict time-series using a large number of heterogeneous training datasets.
deep-learning deep-neural-networks recurrent-neural-networks rnn adam-optimizer time-series random-forest-regressor rmsprop adagrad adamax leaky-relu pca-analysis
Language:Python 26
harshraj11584 / Paper-Implementation-Overview-Gradient-Descent-Optimization-Sebastian-Ruder
[Python] [arXiv/cs] Paper "An Overview of Gradient Descent Optimization Algorithms" by Sebastian Ruder
adadelta adagrad adam-optimizer adam gradient-descent gradient-boosting gradient-descent-algorithm optimization-algorithms optimization-methods adamax amsgrad batch-gradient-descent stochastic-gradient-descent stochastic-optimization stochastic-optimizers momentum nadam nesterov-accelerated-sgd nesterov-momentum rmsprop
Language:Python 25
Janus-Shiau / lookahead_tensorflow
Lookahead optimizer ("Lookahead Optimizer: k steps forward, 1 step back") for tensorflow
tensorflow optimizer adam-optimizer sgd-optimizer deep-learning
Language:Python 25
theroyakash / Adam
Implementation of Adam Optimization algorithm using Numpy
deep-learning adam-optimizer machine-learning algorithms python3 numpy numpy-neural-network
Language:Jupyter Notebook 21
thetechdude124 / Adam-Optimization-From-Scratch
📈Implementing the ADAM optimizer from the ground up with PyTorch and comparing its performance on six 3-D objective functions (each progressively more difficult to optimize) against SGD, AdaGrad, and RMSProp.
adam-optimizer artificial-intelligence deep-learning optimization-algorithms rmsprop sgd-optimizer
Language:Jupyter Notebook 21
declanoller / haskell-vae
Learning about Haskell with Variational Autoencoders
adam-optimizer autoencoder deep-learning dogs facial-landmarks facial-recognition functional-programming haskell hmatrix mnist neural-network variational-autoencoder variational-inference
Language:Haskell 19
jaepil / geometric-adam
A Ray Tracing-Inspired Approach to Neural Network Optimization
adam-optimizer deep-learning deeplearning llms optimizers transformers
Language:Python 17
SSQ / Coursera-Ng-Improving-Deep-Neural-Networks-Hyperparameter-tuning-Regularization-and-Optimization
Short description for quick search
initialization gradient-check l2-regularization dropout mini-batch-gradient-descent exponentially-weighted-averages bias-correction momentum rmsprop adam-optimizer learning-rate-decay batch-normalization softmax-regression tensorflow
Language:Python 17
harshalmittal4 / Hypergradient_variants
Improved Hypergradient optimizers for ML, providing better generalization and faster convergence.
hypergradient optimizers learning-rate step-size adam-optimizer momentum
Language:Jupyter Notebook 16
Gunale0926 / Grams
Grams: Gradient Descent with Adaptive Momentum Scaling (ICLR 2025 Workshop)
machine-learning optimization pytorch adam-optimizer deep-learning optimizer
Language:Python 15
kcg2015 / DDPG_numpy_only
Implemenation of DDPG with numpy only (without Tensorflow)
reinforcement-learning-algorithms actor-critic-algorithm policy-gradient deep-deterministic-policy-gradient neural-network replay-buffer target-network stochastic-gradient-descent adam-optimizer
Language:Python 14
nadir
OptimalFoundation / nadir
Nadir: Cutting-edge PyTorch optimizers for simplicity & composability! 🔥🚀💻
adabelief adam-optimizer adamax adamw optimization radam machine-learning pytorch lion adamp amsgrad sgd-optimizer
Language:Python 14
shashank1623 / Plant-disease-Detection
Plant Disease Detection using convolutional neural network. Our model can easily predict the disease of plants like Potato , Tomato , Pepper Bel and many more in the upcoming version.
adam-optimizer cnn d deep-learning development disease-prediction full-stack gcp mlops neural-network plant plantdiseasedetection prediction web yolov5 collaborate
Language:JavaScript 14
NinaadRao / Multilabel-Image-Classification-using-Contractive-Autoencoder
Implementing contractive auto encoder for encoding cloud images and using that encoding for multi label image classification
contractive-autonencoders convolutional-neural-network feedforward-neural-network adam-optimizer batch-normalization dropout multilabel-classification binary-crossentropy tensorflow keras
Language:Jupyter Notebook 12
rsmath / Digit-Recognizer
A project I made to practice my newfound Neural Network knowledge - I used Python and Numpy to train a network to recognize MNIST images. Adam and mini-batch gradient descent implemented
adam-optimizer deep-learning digit-recognizer gradient-descent neural-network numpy pandas python3
Language:Python 12
thieu1995 / MetaPerceptron
MetaPerceptron: A Standardized Framework For Metaheuristic-Driven Multi-layer Perceptron Optimization
classification-models genetic-algorithm global-search metaheuristic-algorithms metaheuristics mlp multi-layer-perceptron nature-inspired-optimization neural-network particle-swarm-optimization regression-models whale-optimization-algorithm gradient-free-based-multi-layer-perceptron metaheuristic-based-multi-layer-perceptron adagrad-approach adam-optimizer sgd-optimizer adelta-optimizer
Language:Python 12
AdamYuan / SimpleNN
a simple neural network
neural-network adam-optimizer he-initializer mnist-handwriting-recognition sfml
Language:C++ 11
Niranjankumar-c / GradientDescent_Implementation
Implement different variants of gradient descent in python using numpy
adam-optimizer gradient-descent numpy optimization-algorithms python3 sgd-momentum
Language:Jupyter Notebook 11
Chirag-Shilwant / One-Shot-Classification-using-Siamese-Network-on-MNIST-Dataset
A Siamese Neural Network is a class of neural network architectures that contain two or more identical subnetworks. ‘identical’ here means, they have the same configuration with the same parameters and weights.
siamese-network one-shot-learning mnist-dataset keras loss-functions adam-optimizer rmsprop
Language:Jupyter Notebook 10
Emperor-WS / PyEmber
An Educational Framework Based on PyTorch for Deep Learning Education and Exploration
adam-optimizer artificial-intelligence autograd backpropagation convolutional-neural-networks cupy deep-learning deep-learning-framework deep-learning-library deep-neural-networks feedforward-neural-network genetic-algorithm gradient-descent neural-network numpy optimizer perceptrons python pytorch training-algorithms
Language:Python 10

adam-optimizer

LiyuanLucasLiu / RAdam

parasdahal / deepnet

SirRob1997 / Crowded-Valley---Results

tirtharajdash / CS-F425_Deep-Learning

YanaiEliyahu / AdasOptimizer

nengwp / Lion-vs-Adam

aditya9211 / SVHN-CNN

v-iashin / CS231n

yashkant / padam-tensorflow

sagarvegad / Adam-optimizer

shreyansh26 / ML-Optimizers-JAX

Arko98 / Gradient-Descent-Algorithms

VivianoRiccardo / Learning-Lab-C-Library

aromanro / MachineLearning

rdspring1 / Count-Sketch-Optimizers

RudreshVeerkhare / CustomXGBoost

yaricom / TimeSeriesLearning

harshraj11584 / Paper-Implementation-Overview-Gradient-Descent-Optimization-Sebastian-Ruder

Janus-Shiau / lookahead_tensorflow

theroyakash / Adam

thetechdude124 / Adam-Optimization-From-Scratch

declanoller / haskell-vae

jaepil / geometric-adam

SSQ / Coursera-Ng-Improving-Deep-Neural-Networks-Hyperparameter-tuning-Regularization-and-Optimization

harshalmittal4 / Hypergradient_variants

Gunale0926 / Grams

kcg2015 / DDPG_numpy_only

OptimalFoundation / nadir

shashank1623 / Plant-disease-Detection

NinaadRao / Multilabel-Image-Classification-using-Contractive-Autoencoder

rsmath / Digit-Recognizer

thieu1995 / MetaPerceptron

AdamYuan / SimpleNN

Niranjankumar-c / GradientDescent_Implementation

Chirag-Shilwant / One-Shot-Classification-using-Siamese-Network-on-MNIST-Dataset

Emperor-WS / PyEmber