loss-landscape

There are 2 repositories under loss-landscape topic.

• xxxnell / how-do-vits-work

  (ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"

  loss-landscapepytorchself-attentiontransformervision-transformer
  Language:Python 792

• logancyang / loss-landscape-anim

  Create animations for the optimization trajectory of neural nets

  neural-netsoptimization-trajectoryloss-landscapepcapytorchpytorch-lightning
  Language:Python 123

• sayakpaul / Sharpness-Aware-Minimization-TensorFlow

  Implements sharpness-aware minimization (https://arxiv.org/abs/2010.01412) in TensorFlow 2.

  generalizationcomputer-visiondeep-neural-networksloss-landscapetpu-accelerationtensorflow
  Language:Jupyter Notebook 60

• ayulockin / LossLandscape

  Explores the ideas presented in Deep Ensembles: A Loss Landscape Perspective (https://arxiv.org/abs/1912.02757) by Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan.

  deep-neural-networksneural-networksloss-landscapetensorflowkeras
  Language:Jupyter Notebook 58

• shamsbasir / investigating_mitigating_failure_modes_in_pinns

  This repository contains the code and models for our paper "Investigating and Mitigating Failure Modes in Physics-informed Neural Networks(PINNs)"

  adaptive-optimizerconstrained-optimizationdifferential-equationslagrange-multipliersloss-landscapeneural-networksunconstrained-optimization
  Language:Jupyter Notebook 14

• VITA-Group / LTH-Pass

  [TMLR] "Can You Win Everything with Lottery Ticket?" by Tianlong Chen, Zhenyu Zhang, Jun Wu, Randy Huang, Sijia Liu, Shiyu Chang, Zhangyang Wang

  lottery-ticket-hypothesisgeneralizationout-of-distribution-detectionadversarial-robustnessrobustnessuncertaintyflatnessloss-landscapepac-bayesexplanabilityinterpretabilitywinning-tickets
  Language:Python 10

• gg-dema / Git_merge

  analytic solution to the git-merge algorithm, derived from "Git Re-Basin: Merging Models modulo Permutation Symmetries"

  loss-landscapeneural-networks-optimizationpytorch
  Language:Jupyter Notebook 4

• HJHGJGHHG / Optimizer-papers

  Worth-reading papers and related awesome resources on deep learning optimization algorithms. 值得一读的深度学习优化器论文与相关资源。

  convergence-analysisloss-landscapeoptimization-algorithms
  Language:Python 4

• sungyoon-lee / LossLandscapeMatters

  [NeurIPS 2021] Towards Better Understanding of Training Certifiably Robust Models against Adversarial Examples | ⛰️⚠️

  adversarial-attacksadversarial-machine-learningadversarial-examplesrobustnessrobustness-verificationcertified-trainingcertified-defenseloss-landscapesmoothness
  Language:Python 4

• mortfer / keras-gsam

  Surrogate Gap Guided Sharpness-Aware Minimization (GSAM) implementation for keras/tensorflow 2

  deep-learninggsamkerasoptimizersamtensorflowloss-landscape
  Language:Python 2

• pxl-th / Yama.jl

  Visualize loss landscape

  visualizationloss-landscapeneural-networksjuliaflux
  Language:Julia 1