Inference Time
jyotsna2893 opened this issue · comments
Hello, I am using MUSS in google colab notebook, earlier it was taking 3-4 minutes to simplify after downloading the model. I tried it 2 days back and still time taken is ~35 seconds. Is it the usual inference time or there is something I am missing?
Hi,
Yes the inference time using a GPU is about ~40 seconds, however 90% of the time is spent loading the model which is quite big.
As a result simplifying 1 sentence or 100 will take roughly the same time.