Wrong about the computational complexity

[Lethe]

Here, for dense layers, the computational complexity should be: d^2HW(C/2) + b^2HW(C/2) + 3HWC^2

[Zhengzhong Tu]

Given the input feature H,W,C, the whole Multi-axis gMLP block (figure 3) has: Input proj (2HWC^2) + Output proj (2HWC^2) + Block-gMLP dense (3HWC^2) + Grid-gMLP dense (3HWC^2) = 10HWC^2

Note: the first Dense layer in gMLP block expands channels from C to 2C, thus 2HWC^2. Output Dense in gMLP is HWC^2. So the whole gMLP block has 3HWC^2.

[Lethe]

I see. Thank you. But, the figure 3 is confused with C and C / 2.

[Zhengzhong Tu]

Oh yeah you're right. We follow common complexity convention from Swin, etc, but didn't expect to cause a confusion.