Is there a batched-based keyword extraction approach with keyBERT?
celsofranssa opened this issue · comments
I am extracting keywords from 2.5 million sentences, iterating through the list of sentences. However, this approach is only using 8% of the GPU.
Is there a batched-based keyword extraction approach with keyBERT?
@celsofranssa Yes, you can pass a list of sentences at each call to keybert, and then retrieve the keywords for each sentence in the same index position.
@celsofranssa It is difficult to say what is happening here. 8% is tricky to interpret without knowing the full context. For instance, does that relate to the embedding step of the process (which you can perform manually beforehand)? Also, it depends on the underlying embedding model that you use. Without seeing the code, it is difficult to say.