Should non-trivial irreps support bias or not?

[Bo Li]

Hi @Gabri95 ,

I noticed the current implementation only supports bias for trivial representation as commented:

e2cnn/e2cnn/nn/modules/r2_conv/r2convolution.py

Lines 184 to 189 in 2604f67

 if bias: 
     # bias can be applied only to trivial irreps inside the representation 
     # to apply bias to a field we learn a bias for each trivial irreps it contains 
     # and, then, we transform it with the change of basis matrix to be able to apply it to the whole field 
     # this is equivalent to transform the field to its irreps through the inverse change of basis, 
     # sum the bias only to the trivial irrep and then map it back with the change of basis 

From a naive intuition, I feel that bias acted on the norm of an irrep seems also equivariant. I didn't find relevant description in the paper. So just a short question for any relevant reference to deny this intuition. Thanks!

[Gabriele Cesa]

Hi @prclibo

Thanks for your question!

indeed, the bias can only be added to the trivial representations.
One can show that adding a bias to any other irreducible representation would break equivariance.

However, what you suggest is not the usual notion of bias.
Usually, a linear/conv layer performs an affine transformation, while the bias applied to the norm is a non-linear operation.
For this reason, we chose to not include it in the convolution layer.
As we only needed this kind of bias in the implementation of HNet with norm-nonlinearities we chose to add this bias directly in the implementation of the non-linearity, see https://quva-lab.github.io/e2cnn/api/e2cnn.nn.html#e2cnn.nn.NormNonLinearity

If you need to this kind of bias outside the non-linearity, I can add it as a separate module

Gabriele

[Bo Li]

Hi @Gabri95

Thanks for the reply! This explanation makes sense to me. It is fine to use nonlinearity instead.

Bo