cauchy-schwarz-inequality

There are 0 repository under cauchy-schwarz-inequality topic.

• brandontang892 / quant_aware_L2Grad_regularization

  Our work implements novel L2-Norm gradient (L2Grad) and variance of the weight distrbution (VarianceNorm) regularizers for quantization-aware training such that the distribution of weights are more compatible with post-training quantization especially for low bit-widths. We provide a theoretical basis that directly relates L2-Grad with post quantization test accuracy through a first order Taylor Series expansion followed by the reduction to an adversary with an L2 budget, in which we apply the Cauchy-Schwarz inequality to provide the desired bounds. We empirically show that L2Grad and VarianceNorm can both match the performance of L1Grad and outperform it on certain bit-widths. We also show that a regularization scheme that combines L2Grad and VarianceNorm in a novel "regularization scheduling" methodology can give even better results in terms of post-quantization accuracy, tested on uniform and piecewise linear quantization.

  quantization-aware-trainingregularizationpytorchcnnwandbcauchy-schwarz-inequality
  Language:Jupyter Notebook 1

• CarlosLunaMota / InequalitiesCheatSheet

  A cheat sheet with five useful inequalities

  inequalitiescheat-sheetrearrangement-inequalitycauchy-schwarz-inequalityam-gm-inequalityholder-inequalityjensen-inequality
  Language:TeX 1