How do I calculate the FLOPs of a model with some frozen layers during training?

[Jungseob Lee]

Thank you so much for sharing the code.

As far as I know, frozen layers have no effect on FLOPs in the forward pass. That is, even if frozen layers are included, FLOPs is only affected by the total number of parameters.

So, how do estimate the FLOPs in the backward pass when there are some frozen layers im the model? Is it correct to simply calculate 2 * forward_FLOPs?

I wonder if this code reflects my question.
If not, can someone please help me?

[juliannadvl]

Hi, have you figured it out ?