kermitt2 / delft

a Deep Learning Framework for Text

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Find a way to disable the ELMo/BERT caching mechanism in "production" mode

oterrier opened this issue · comments

The contextual embeddings (ELMo or BERT) caching mechanism using a lmdb database is really nice especially in training mode because it saves a lot of time after the 1st epoch.
Anyway when you want to use intensively your trained model in "production" making a lot of prediction over a long period of time, the lmdb database can potentially grows infinitively when providing massively unseen texts to the Tagger.tag() method
It could be usefull to have a way of disabling the cache, like for example with a use_cache boolean flag that could overwrite the default behaviour (not during training but in production)

Not sure that it is crystal clear ... Let me know

Thank you @oterrier, this is very clear and actually using the lmdb cache for ELMo and BERT embeddings was only intended for training. I think for prediction it never makes no sense to use it (it can very quickly consume too much disk space) and should always been deactivated. A use_cache boolean is indeed necessary !