How to run FA

[Mrigank Raman]

Thanks for supporting FA!! I was wondering where to find the code changes needed to be done to use FA

[Minh-Thuc]

You just set the flash_attention parameter when create the Generator:

generator = ctranslate2.Generator(model_dir, device="cuda", flash_attention=True)

[BBC-Esq]

You can also go to the documents and search for "flash" and it'll give you links to how to use it for the "generator" as well as other parts of CT2!