chrisjbryant / lmgec-lite

A language model-based approach to Grammatical Error Correction for English that uses minimal annotated data.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Make loading huge KenLM models better when handling huge LM models

gurunathparasaram opened this issue · comments

When loading KenLM models of huge size(in my case tens of GB), the present method handling the loading of KenLM models is slow, takes up more RAM and makes the process crash. I think we can load the models in lazy way and make the model loading quicker and memory-efficient.

This seems more like it depends on what resources you have available.

KenLM actually recommends loading the whole model into memory if you have the resources (link), but it's good to know that there is also a lazy way to load the model if you don't!

Would adding a flag like -lazy to load the models lazily help so that the default behaviour (of loading the whole model into memory) doesn't get changed?