Giters

mit-han-lab / lite-transformer

[ICLR 2020] Lite Transformer with Long-Short Range Attention

Home Page:https://arxiv.org/abs/2004.11886

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to measure the FLOPs/MACs?

ranery opened this issue · comments

commented

Hi, thanks for your great work!

I am curious how you measure the flops/macs in your reported numbers, will the torchprofile package handle the measurement of the customized Cuda code?

Best,
Haoran

commented

Thank you for asking! We use the standard PyTorch code to measure the MACs, instead of the customized cuda code, so all of the computation of the ffn and attention/conv module is included in the reported number.

commented

Thank you for asking! We use the standard PyTorch code to measure the MACs, instead of the customized cuda code, so all of the computation of the ffn and attention/conv module is included in the reported number.

can you share the coder which calculate the (Mult-Adds and flop？