Make loading huge KenLM models better when handling huge LM models
gurunathparasaram opened this issue · comments
When loading KenLM models of huge size(in my case tens of GB), the present method handling the loading of KenLM models is slow, takes up more RAM and makes the process crash. I think we can load the models in lazy way and make the model loading quicker and memory-efficient.
This seems more like it depends on what resources you have available.
KenLM actually recommends loading the whole model into memory if you have the resources (link), but it's good to know that there is also a lazy way to load the model if you don't!
Would adding a flag like -lazy
to load the models lazily help so that the default behaviour (of loading the whole model into memory) doesn't get changed?