FP16 and NVIDIA Apex support

[Nic Ma]

Hi @king-menin ,

Do you support FP16 and NVIDIA Apex as the Pytorch BERT examples?
Thanks.

[Anton Emelyanov]

You can rewrite train.py or try to replace learner.optimizer to your own favorite optimizer.

PS
I am alone on this project, so add FP16 was in my task list too, but I have many other task from bosses :-) If you can write, do merge request plz.

[Nic Ma]

@king-menin

Thanks, I will try to add FP16 support with NVIDIA Apex lib later.