Loss parameters optimization
mnikitin opened this issue · comments
Hello!
Thanks for your work, very interesting reformulation of margin-based softmax !
My question is why to use REINFORCE for searching of a
?
Can it be optimized together with parameters of the network, i.e. updated at each training iteration?
Probably, it's obvious question, sorry for that.
Thanks for your comments. It is meaningful to optimize the hyperparameter together with parameters of the network. In this paper, we first explore that the essence of margin-based softmax losses is to reduce the softmax probability and then propose the Random-Softmax. For searching the loss function, we follow the strategy of AM-LFS but define a new search space.