This model use a lots of memory

[Bingyu.Jiang]

Beside bert model, after I train my model.
Load bert and trained model, also data, it need 4.5G, this is very large.
And during this situation, very hard to deploy online.
So is there anyway to reduce memory use?

[Anton Emelyanov]

reduce number of your parametres ))

[Anton Emelyanov]

only BERT embedder need about 2 G gpu memory (in eval mode, without any additional layers on top).

[Bingyu.Jiang]

I only add one lstm layer with 256 hiden states and a crf layer.

[Paul Zhang Yixing]

it is the reality. Otherwise you have to reduce the parameters