OpenNMT / CTranslate2

Fast inference engine for Transformer models

Home Page:https://opennmt.net/CTranslate2

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[INFO] cuDNN with fusions

mnicely opened this issue · comments

Hi, I came across this repo while wiring up Whisper support to my Bazaar. I just wanted to say amazing repo!

Diving through the code I noticed there was a mixture of cuBLAS and cuDNN. I wanted to let you know we just released cuDNN v9 and the new C++/Python Frontends. The new Graph API allows developers to fuse many combinations of ops such as Matmul + pointwise. If you're interested in more information, let me know. Again, super cool work!

Hello,
Thank you for your information. Currently, we don't plan to work on it, but we will see it more in detail.