Embedding generation runs on CPU only
rti opened this issue · comments
Terms
- I have searched all open bug reports
- I agree to follow Wikimedia's Code of Conduct
Behavior
When generating embeddings, only CPU is used, no GPU acceleration is leveraged.
This makes embedding generation for our full example data requiring 18h on 16 cores.
Typically, GPU acceleration can be activated by providing a device="cuda" parameter. This should speed up the embedding generation.
Operating System
linux, our container on runpod.io host with nvidia 3090