How to use transfer learning using 5k iteration model?

Question

How to use transfer learning using 5k iteration model?

Nishanksingla opened this issue 7 years ago · comments

I have limited GPU resources and I can not train the model for so long.
I kept it for one whole day for training and I was only able to reach to 300 something iterations(in between 2nd epoch).

Can anyone please tell me how much time it will take to train the complete model and is it possible to do transfer learning using 5k iteration model?

Also if anyone is able to train the complete model and got the good results, can you please share the model? I kind off require it for my academic project.

Thanks in advance :)

Scott Stephenson · Answer 1 · Wed Apr 12 2017 02:13:38 GMT+0800 (China Standard Time)

@Nishanksingla It depends on what your goals are. Speech recognition using DNNs is very much a research area that is compute and data limited.

The state of Speech A.I. is not like that of Image A.I. If you want a tiny word error rate (like <5%) on a wide range of audio (phone calls, radio shows, youtube vids, recorded lectures), then that doesn't exist yet. If you want to have some fun and see the tech working pretty well (but not perfectly) and be part of the progress as things get better, then you can do that with even limited resources. The best way it by contributing to open source deep learning projects like this one. There still is a lot of work to do.

To give some concrete advice. What type of GPU do you have access to? If it is something like a K80 (7 TFLOPS and 12GB of memory), then you should be able to get an 'ok model' in a day. If you want a 'pretty good model on a specific dataset' then you could get that by running for 2-4 weeks (with the full 1000 hour LS dataset). If is is something more limited (like 2TFLOPS and 4GB) then you can still make a dent, but you probably won't be happy with the progress unless you are really patient and happy with marginal improvements per week.

Nishank Singla · Answer 2 · Wed Apr 12 2017 02:26:59 GMT+0800 (China Standard Time)

Hi Scott

Thank you for the reply.
My goals are simple, just want to train this model on LibriSpeech dataset to see how well it can perform on my recorded voice.

I am interested in contributing to this project. Please let me know how can I contribute and from where I can get the information about it

Scott Stephenson · Answer 3 · Wed Apr 12 2017 02:28:42 GMT+0800 (China Standard Time)

Which GPU do you have?

P.S. I'm not at Baidu so my advice is just coming from a fellow person interested in Speech A.I. :)