leaderj1001 / Stand-Alone-Self-Attention

Implementing Stand-Alone Self-Attention in Vision Models using Pytorch

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

problem with unfold

vainaixr opened this issue · comments

after

k_out = k_out.contiguous().view(batch, self.groups, self.out_channels // self.groups, height, width, -1)

it gives error,

RuntimeError: shape '[2, 1, 16, 34, 34, -1]' is invalid for input of size 294912

this is because of use of unfold on k_out before this, so height, width is not consistent.

You should change the padding to 1, so the dimension w and h of the feature maps would keep the same after 1x1 convolusion.

class AttentionConv(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=1, groups=1, bias=False):