Freeze network parameters
gwario opened this issue · comments
Hi!
As I need to freeze some of my network's parameters, i was trying to implement Network.params to return only the parameters i actually want to be optimized.
However this seems to not do what i thought since Network.train! does call (in the case of Flux) Flux.params on the network and not my version of Network.params so if this is the intended behavior, I need to override network.train!... if it is a mistake, the fix is probably to replace Flux.params with Network.params for Flux, for Knet I'm not sure how to do it though.
Could you please clarify
Thanks!
Regards GW
I think this is a bug indeed and Network.train! should not be calling Flux directly.
Feel free to submit a PR, especially if you can figure out how to implement params for Knet!
Hm i it works for flux when changing it to Network.params .. for knet i don't know sadly... think it is good to change that behavior for both network libs at the same time... what do you think? or should i just submit the change for flux?
Does this mean that the Knet backend is currently broken?
I have a hard time imagining things going well if Flux.params is called on a Knet network.
Honestly though, supporting both Flux and Knet is a maintenance nightmare and the cause of much useless complexity in the codebase. I wish there was a better story for compatibility between ML frameworks (and I know Deniz Yuret has been doing some work toward this) but this should not be AlphaZero.jl's responsibility.