About the code of re-initializing the top layer(s) before fine-tuning

Question

About the code of re-initializing the top layer(s) before fine-tuning

LYPinASR opened this issue a year ago · comments

Hello!
Could you please show the code of re-initializing the top layer(s) before fine-tuning?
Thank you!

Ju-Chieh Chou · Answer 1 · Tue Apr 11 2023 00:40:22 GMT+0800 (China Standard Time)

Hi, the fine-tuning is done separately. I use speechbrain to fine-tune with this pytorch module, but with the module, it should be easy to fine-tune it without speechbrain (can be done in huggingface I believe). My huggingface transformers version is (4.6.0.dev0). You can find the code for re-init in L185-191.

LYPinASR · Answer 2 · Thu Apr 27 2023 08:25:45 GMT+0800 (China Standard Time)

Thanks for your reply! However, the link of "pytorch module" is invalid for me. Could you please send me the corresponding codes?

Ju-Chieh Chou · Answer 3 · Thu Apr 27 2023 09:55:57 GMT+0800 (China Standard Time)

Hi, I just make it public. Please try again.