wrapping pytorch operations - grid_sample
ahyunSeo opened this issue · comments
Hello,
Thank you for your nice work!
I'm about to use the e2cnn based network for my project.
I need to use the method grid_sample in PyTorch and its underlying implementation is just some interpolation.
https://pytorch.org/docs/stable/generated/torch.nn.functional.grid_sample.html?highlight=grid_sample#torch.nn.functional.grid_sample
I noticed the upsample in this repo simply computes the value and reassigns the output type.
https://github.com/QUVA-Lab/e2cnn/blob/master/e2cnn/nn/modules/r2upsampling.py#L86
I think I can do exactly like this with grid_sample.
Please let me know if there will be any issues with this.
Ahyun
Hi @ahyunSeo
What do you need grid-sample for?
Wrapping it in an EquivariantModule (as done in r2upsampling) is indeed not a problem from an implementation point of view.
However, grid_sample can perform some arbitrary deformation of the input, which means it is not generally equivariant and, therefore, should not be implemented in a subclass of EquivariantModule.
Are you maybe assuming some specific grid which guarantees equivariance?
Best,
Gabriele
Hi @Gabri95
You made a fair point.
I was about to sample a circular/polar grid for each pixel, where each pixel becomes its center.
So each pixel will be assigned with # angle of grid points.
Then, I wanted to compute correlation values between the grid points per pixel for the further pipeline.
Maybe I should move the feature tensor to a non-geometric tensor first.
Regards,
Ahyun
Hi @ahyunSeo ,
If the operation you are trying to implement is equivariant you can just wrap it in an EquivariantModule as I did for r2upsampling.
In the forward pass, you just extract the 'tensor' from the GeometricTensor input, process this tensor is an equivariant way and finally wrap the output in a new GeometricTensor.
You can precompute the grid in the init of the module such that you can ensure the grid allows equivariance.
Let me know if this helps
Gabriele
Thanks a lot!