spectral_normalization_nondiff.py

Question

spectral_normalization_nondiff.py

hellopipu opened this issue 5 years ago · comments

Thanks for your code . I'm quite confused with the nondiff version, what does it for , and what's the difference with the diff version. Thanks

Christian Cosgrove · Answer 1 · Tue May 28 2019 07:10:19 GMT+0800 (China Standard Time)

nondiff stands for nondifferentiable. Rather than creating a new tensor to represent the normalized weights (which is what the other version does), the nondiff version simply replaces w with a normalized weight matrix. In my experiments (long time ago) I found there wasn’t much of a difference between these two implementations.

…

On May 26, 2019, at 8:48 PM, Bingyu Xin ***@***.***> wrote: Thanks for your code . I'm quite confused with the nondiff version, what does it for , and what's the difference with the diff version. Thanks — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.