equivariance in C8 space

[ABHISHEK PERI]

Hi,

When I'm testing the equivariance for a sample example in C8 space following the "introduction.ipynb", I've observed it to fail. So, I wanted to check if there is a correct way of evaluating the c8 space equivariance? Further, It would help me if you could point out how could one go about defining an architecture such that equivariance could be observed across all angles of C8 space?

My code:

import torch

from e2cnn import gspaces
from e2cnn import nn


def conv3x3(in_type: nn.FieldType, out_type: nn.FieldType, stride=1, padding=1,
            dilation=1, bias=False):
    """3x3 convolution with padding"""
    return nn.R2Conv(in_type, out_type, 3,
                      stride=stride,
                      padding=padding,
                      dilation=dilation,
                      bias=bias,
                      sigma=None,
                      frequencies_cutoff=lambda r: 3*r,
                      )

def conv1x1(in_type: nn.FieldType, out_type: nn.FieldType, stride=1, padding=0,
            dilation=1, bias=False):
    """1x1 convolution with padding"""
    return nn.R2Conv(in_type, out_type, 1,
                      stride=stride,
                      padding=padding,
                      dilation=dilation,
                      bias=bias,
                      sigma=None,
                      frequencies_cutoff=lambda r: 3*r,
                      )


r2_act = gspaces.Rot2dOnR2(N=8)
feat_type_in = nn.FieldType(r2_act, [r2_act.trivial_repr])
feat_type_out = nn.FieldType(r2_act, 3*[r2_act.regular_repr])

conv = conv3x3(feat_type_in, feat_type_out)

x = torch.randn(4, 1, 32, 32)
x = nn.GeometricTensor(x, feat_type_in)

assert isinstance(x.tensor, torch.Tensor)

y = conv(x)

assert y.type == feat_type_out

print(y.tensor.shape)


for g in r2_act.testing_elements:
    # transform the input with the current group element according to the input type
    x_transformed = x.transform(g)
    
    # feed the transformed input in the convolutional layer
    y_from_x_transformed = conv(x_transformed)
    
    # the result should be equivalent to rotating the output produced in the 
    # previous block according to the output type
    y_transformed_from_x = y.transform(g)
    
    assert torch.allclose(y_from_x_transformed.tensor, y_transformed_from_x.tensor, atol=1e-05), g

print("success")

result:

Traceback (most recent call last):
  File "e2_test.py", line 62, in <module>
    assert torch.allclose(y_from_x_transformed.tensor, y_transformed_from_x.tensor, atol=1e-05), g
AssertionError: 1

[Gabriele Cesa]

Hi @abhishek-peri

Unfortunately, perfect equivariance to C8 is not possible since that includes equivariance to rotations by 45 deg, which is not exact for a pixel grid.
In other words, equivariance to C8 is only approximate and comparing the outputs of the convolution is too sensitive to the interpolation artifacts (and an absolute tollerance of atol=1e-05 is probably too low).

To evaluate the equivariance error, I'd recommend starting with higher resolution images, rotate them in this higher resolution and then downsample them to reduce the artifacts.
Then, perform convolution at this resolution.
Finally, rotate one of the two outputs and further downsample the outputs to reduce the interpolation artifacts.
This is what I do in my unit tests (see here and here.

Hope this helps,

Gabriele