Congrats on flash attention, now how do I run it???

Question

Congrats on flash attention, now how do I run it???

BBC-Esq opened this issue 3 months ago · comments

Saw that the release finally uploaded to pypi and am excited to test FA...how do I do it? I saw instructions somewhere about setting a boolean parameter??

BBC-Esq · Answer 1 · Thu Apr 25 2024 00:35:16 GMT+0800 (China Standard Time)

Nevermind, found it.

Mrigank Raman · Answer 2 · Thu Apr 25 2024 15:07:35 GMT+0800 (China Standard Time)

Nevermind, found it.

Can you help me find it?

BBC-Esq · Answer 3 · Thu Apr 25 2024 15:47:09 GMT+0800 (China Standard Time)

Nevermind, found it.

Can you help me find it?

https://opennmt.net/CTranslate2/python/ctranslate2.Generator.html?highlight=flash