Support converting HF reward models to .nemo
odelalleau opened this issue · comments
Is your feature request related to a problem? Please describe.
Converting a HF reward model to .nemo doesn't seem to work right now. See discussion in #109 for details.
Describe the solution you'd like
We should have a script to convert HF reward models to .nemo.
Hi Olivier,
Thanks for this. Also we would like to have a backward script which can convert .nemo RM to HF.