Giters

ZikangZhou / awesome-equivariant-network

Paper list for equivariant neural network

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

awesome-equivariant-network

Paper list for equivariant neural network. Work-in-progress.

Feel free to suggest relevant papers in the following format.

**Group Equivariant Convolutional Networks**  
Taco S. Cohen, Max Welling ICML 2016 [paper](https://arxiv.org/pdf/1602.07576.pdf)

Acknowledgement: I would like to thank Maurice Weiler, Fabian Fuchs, Tess Smidt, Rui Wang, David Pfau, Jonas Köhler, Taco Cohen, Gregor Simm, Erik J Bekkers, Jean-Baptiste Cordonnier, David W. Romero, Ivan Sosnovik, Kostas Daniilidis for paper suggestions! Thank Weihao Xia for helping out typesetting!

Table of Contents

Equivariance and Group convolution

  1. Group Equivariant Convolutional Networks
    Taco S. Cohen, Max Welling ICML 2016 paper
    Note: first paper; discrete group;

  2. Steerable CNNs
    Taco S. Cohen, Max Welling ICLR 2017 paper

  3. Harmonic Networks: Deep Translation and Rotation Equivariance
    Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, Gabriel J. Brostow CVPR 2017 paper

  4. Spherical CNNs
    Taco S. Cohen, Mario Geiger, Jonas Koehler, Max Welling ICLR 2018 best paper paper
    Note: use generalized FFT to speed up convolution on $S^2$ and $SO(3)$

  5. Clebsch–Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network
    Risi Kondor, Zhen Lin, Shubhendu Trivedi NeurIPS 2018 paper
    Note: perform equivariant nonlinearity in Fourier space;

  6. General E(2)-Equivariant Steerable CNNs
    Maurice Weiler, Gabriele Cesa NeurIPS 2019 paper
    Note: nice benchmark on different reprsentations

  7. Learning Steerable Filters for Rotation Equivariant CNNs
    Maurice Weiler, Fred A. Hamprecht, Martin Storath CVPR 2018 paper
    Note: group convolutions, kernels parameterized in circular harmonic basis (steerable filters);

  8. Learning SO(3) Equivariant Representations with Spherical CNNs
    Carlos Esteves, Christine Allen-Blanchette, Ameesh Makadia, Kostas Daniilidis ECCV 2018 paper
    Note: SO(3) equivariance; zonal filter

  9. Polar Transformer Networks
    Carlos Esteves, Christine Allen-Blanchette, Xiaowei Zhou, Kostas Daniilidis ICLR 2018 paper

  10. 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
    Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, Taco Cohen NeurIPS 2018 paper
    Note: SE(3) equivariance; characterize the basis of steerable kernel

  11. Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
    Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, Patrick Riley paper
    Note: SE(3) equivariance for point clouds

  12. Equivariant Multi-View Networks
    Carlos Esteves, Yinshuang Xu, Christine Allen-Blanchette, Kostas Daniilidis ICCV 2019 paper

  13. Gauge Equivariant Convolutional Networks and the Icosahedral CNN
    Taco S. Cohen, Maurice Weiler, Berkay Kicanaoglu, Max Welling ICML 2019 paper, talk
    Note: gauge equivariance on general manifold

  14. Cormorant: Covariant Molecular Neural Networks
    Brandon Anderson, Truong-Son Hy, Risi Kondor NeurIPS 2019 paper

  15. Deep Scale-spaces: Equivariance Over Scale
    Daniel Worrall, Max Welling NeurIPS 2019 paper

  16. Scale-Equivariant Steerable Networks
    Ivan Sosnovik, Michał Szmaja, Arnold Smeulders ICLR 2020 paper

  17. B-Spline CNNs on Lie Groups
    Erik J Bekkers ICLR 2020 paper

  18. SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
    Fabian B. Fuchs, Daniel E. Worrall, Volker Fischer, Max Welling NeurIPS 2020 paper, blog
    Note: TFN + equivariant self-attention; improved spherical harmonics computation

  19. Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
    Pim de Haan, Maurice Weiler, Taco Cohen, Max Welling ICLR 2021 paper
    Note: anisotropic gauge equivariant kernels + message passing by parallel transporting features over mesh edges

  20. Lorentz Group Equivariant Neural Network for Particle Physics
    Alexander Bogatskiy, Brandon Anderson, Jan T. Offermann, Marwah Roussi, David W. Miller, Risi Kondor ICML 2020 paper
    Note: SO(1, 3) equivariance

  21. Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
    Marc Finzi, Samuel Stanton, Pavel Izmailov, Andrew Gordon Wilson ICML 2020 paper
    Note: fairly generic architecture; use Monte Carlo sampling to achieve equivariance in expectation;

  22. Spin-Weighted Spherical CNNs
    Carlos Esteves, Ameesh Makadia, Kostas Daniilidis NeurIPS 2020 paper
    Note: anisotropic filter for vector field on sphere

  23. Learning Invariances in Neural Networks
    Gregory Benton, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson NeurIPS 2020 paper
    Note: very interesting approch; enfore "soft" invariance via learning over both model parameters and distributions over augmentations

  24. Lie Algebra Convolutional Neural Networks with Automatic Symmetry Extraction
    Nima Dehmamy, Yanchen Liu, Robin Walters, Rose Yu paper
    Note: very interesting paper; It’s unfortunate that it is rejected by ICLR 2021

  25. LieTransformer: Equivariant self-attention for Lie Groups
    Michael Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Teh, Hyunjik Kim paper
    Note: equivariant self attention to arbitrary Lie groups and their discrete subgroups

  26. Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data
    David W. Romero, Mark Hoogendoorn ICLR 2020 paper

  27. Attentive Group Equivariant Convolutional Networks
    David W. Romero, Erik J. Bekkers, Jakub M. Tomczak, Mark Hoogendoorn ICML 2020 paper

  28. Wavelet Networks: Scale Equivariant Learning From Raw Waveforms
    David W. Romero, Erik J. Bekkers, Jakub M. Tomczak, Mark Hoogendoorn paper

  29. Group Equivariant Stand-Alone Self-Attention For Vision
    David W. Romero, Jean-Baptiste Cordonnier ICLR 2021 paper

  30. MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning
    Elise van der Pol, Daniel E. Worrall, Herke van Hoof, Frans A. Oliehoek, Max Welling NeurIPS 2020 paper

  31. Isometric Transformation Invariant and Equivariant Graph Convolutional Networks
    Masanobu Horie, Naoki Morita, Toshiaki Hishinuma, Yu Ihara, Naoto Mitsume ICLR 2021 paper

  32. E(n) Equivariant Graph Neural Networks
    Victor Garcia Satorras, Emiel Hoogeboom, Max Welling ICML 2021 paper
    Note: a simple alternative that achieves E(n) equivariance

  33. Vector Neurons: A General Framework for SO(3)-Equivariant Networks
    Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, Leonidas Guibas paper
    Note: a simple MLP for type-1 features

  34. Equivariant message passing for the prediction of tensorial properties and molecular spectra
    Kristof T. Schütt, Oliver T. Unke, Michael Gastegger ICML 2021 paper

Theory

  1. On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
    Risi Kondor, Shubhendu Trivedi ICML 2018 paper
    Note: convolution is all you need (for scalar fields)

  2. A General Theory of Equivariant CNNs on Homogeneous Spaces
    Taco Cohen, Mario Geiger, Maurice Weiler NeurIPS 2019 paper
    Note: convolution is all you need (for general fields)

  3. Equivariance Through Parameter-Sharing
    Siamak Ravanbakhsh, Jeff Schneider, Barnabas Poczos ICML 2017 paper

  4. Universal approximations of invariant maps by neural networks
    Dmitry Yarotsky paper

  5. A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
    Leon Lang, Maurice Weiler ICLR 2021 paper
    Note: steerable kernel spaces are fully understood and parameterized in terms of 1) generalized reduced matrix elements, 2) Clebsch-Gordan coefficients, and 3) harmonic basis functions on homogeneous spaces.

  6. On the Universality of Rotation Equivariant Point Cloud Networks
    Nadav Dym, Haggai Maron ICLR 2021 paper,
    Note: universality for TFN and se3-transformer

  7. Universal Equivariant Multilayer Perceptrons
    Siamak Ravanbakhsh paper

  8. Provably Strict Generalisation Benefit for Equivariant Models
    Bryn Elesedy, Sheheryar Zaidi ICML 2021 paper

Equivariant Density Estimation and Sampling

  1. Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities
    Jonas Köhler, Leon Klein, Frank Noé ICML 2020 paper
    Note: general framework for constructing equivariant normalizing flows on euclidean spaces. Instantiation for particle systems/point clouds = simultanoues SE(3) and permutation equivariance.
  2. Equivariant Hamiltonian Flows
    Danilo Jimenez Rezende, Sébastien Racanière, Irina Higgins, Peter Toth NeurIPS 2019 ML4Phys workshop paper
    Note: general framework for constructing equivariant normalizing flows in phase space utilizing Hamiltonian dynamics. Instantiation for SE(2) equivariance.
  3. Sampling using SU(N) gauge equivariant flows
    Denis Boyda, Gurtej Kanwar, Sébastien Racanière, Danilo Jimenez Rezende, Michael S. Albergo, Kyle Cranmer, Daniel C. Hackett, Phiala E. Shanahan paper
    Note: normalizing flows for lattice gauge theory. Instantiation for SU(2)/SU(3) equivariance.
  4. Exchangeable neural ode for set modeling
    Yang Li, Haidong Yi, Christopher M. Bender, Siyuan Shan, Junier B. Oliva NeurIPS 2020 paper
    Note: framework for permutation equivariant flows for set data. Instantiation for permutation equivariance.
  5. Equivariant Normalizing Flows for Point Processes and Sets
    Marin Biloš, Stephan Günnemann NeurIPS 2020 paper
    Note: framework for permutation equivariant flows for set data. Instantiation for permutation equivariance.
  6. The Convolution Exponential and Generalized Sylvester Flows
    Emiel Hoogeboom, Victor Garcia Satorras, Jakub M. Tomczak, Max Welling NeurIPS 2020 paper
    Note: invertible convolution operators. Instantiation for permutation equivariance.
  7. Targeted free energy estimation via learned mappings
    Peter Wirnsberger, Andrew J. Ballard, George Papamakarios, Stuart Abercrombie, Sébastien Racanière, Alexander Pritzel, Danilo Jimenez Rezende, Charles Blundell J Chem Phys. 2020 Oct 14;153(14):144112. paper
    Note: normalizing flows for particle systems on a torus. Instantiation for permutation equivariance.
  8. Temperature-steerable flows
    Manuel Dibak, Leon Klein, Frank Noé NeurIPS 2020 ML4Phys workshops paper
    Note: normalizing flows in phase space with equivariance with respect to changes in temperature.

Application

  1. Trajectory Prediction using Equivariant Continuous Convolution
    Robin Walters, Jinxi Li, Rose Yu ICLR 2021 paper
  2. Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
    Rui Wang, Robin Walters, Rose Yu ICLR 2021 paper
  3. SE(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
    Simon Batzner, Tess E. Smidt, Lixin Sun, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Boris Kozinsky paper
  4. Finding Symmetry Breaking Order Parameters with Euclidean Neural Networks
    Tess E. Smidt, Mario Geiger, Benjamin Kurt Miller paper
  5. Group Equivariant Generative Adversarial Networks
    Neel Dey, Antong Chen, Soheil Ghafurian ICLR 2021 paper
  6. Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks
    David Pfau, James S. Spencer, Alexander G. de G. Matthews, W. M. C. Foulkes paper
  7. Symmetry-Aware Actor-Critic for 3D Molecular Design
    Gregor N. C. Simm, Robert Pinsler, Gábor Csányi, José Miguel Hernández-Lobato ICLR 2021 paper
  8. Roto-translation equivariant convolutional networks: Application to histopathology image analysis
    Maxime W. Lafarge, Erik J. Bekkers, Josien P.W. Pluim, Remco Duits, Mitko Veta MedIA paper
  9. Scale Equivariance Improves Siamese Tracking
    Ivan Sosnovik*, Artem Moskalev*, Arnold Smeulders WACV 2021 paper
  10. 3D G-CNNs for Pulmonary Nodule Detection Marysia Winkels, Taco S. Cohen paper International Conference on Medical Imaging with Deep Learning (MIDL), 2018.
  11. Roto-translation covariant convolutional networks for medical image analysis
    Erik J. Bekkers, Maxime W. Lafarge, Mitko Veta, Koen A.J. Eppenhof, Josien P.W. Pluim, Remco Duits MICCAI 2018 Young Scientist Award paper
  12. Equivariant Spherical Deconvolution: Learning Sparse Orientation Distribution Functions from Spherical Data
    Axel Elaldi*, Neel Dey*, Heejong Kim, Guido Gerig, Information Processing in Medical Imaging (IPMI) 2021 paper
  13. Rotation-Equivariant Deep Learning for Diffusion MRI
    Philip Müller, Vladimir Golkov, Valentina Tomassini, Daniel Cremers paper
  14. Equivariant geometric learning for digital rock physics: estimating formation factor and effective permeability tensors from Morse graph
    Chen Cai, Nikolaos Vlassis, Lucas Magee, Ran Ma, Zeyu Xiong, Bahador Bahmani, Teng-Fong Wong, Yusu Wang, WaiChing Sun paper
    Note: equivariant nets + Morse graph for permeability tensor prediction

Permutation Equivariance

There are many paper on this topics. I only added very few of them.

  1. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
    Charles R. Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas CVPR 2017 paper
  2. Deep Sets
    Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, Alexander Smola NeurIPS 2017 paper
  3. Invariant and Equivariant Graph Networks
    Haggai Maron, Heli Ben-Hamu, Nadav Shamir, Yaron Lipman ICLR 2019 paper
  4. Provably Powerful Graph Networks
    Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, Yaron Lipman NeurIPS 2019 paper
  5. Universal Invariant and Equivariant Graph Neural Networks
    Nicolas Keriven, Gabriel Peyré NeurIPS 2019 paper
  6. On Learning Sets of Symmetric Elements
    Haggai Maron, Or Litany, Gal Chechik, Ethan Fetaya ICML 2020 best paper
  7. On the Universality of Invariant Networks
    Haggai Maron, Ethan Fetaya, Nimrod Segol, Yaron Lipman paper

Talk and Tutorial

IAS: Graph Nets: The Next Generation - Max Welling - YouTube

Equivariance and Data Augmentation workshop: many nice talks

IPAM: Tess Smidt: "Euclidean Neural Networks for Emulating Ab Initio Calculations and Generating Atomi..." - YouTube

IPAM: E(3) Equivariant Neural Network Tutorial

IPAM: Risi Kondor: "Fourier space neural networks"

NeurIPS 2020 tutorial: Equivariant Networks

Yaron Lipman - Deep Learning of Irregular and Geometric Data - YouTube

Math-ML: Erik J Bekkers: Group Equivariant CNNs beyond Roto-Translations: B-Spline CNNs on Lie Groups

Kostas Daniilidis: Geometry-aware deep learning: A brief history of equivariant representations and recent results

Andrew White: Deep Learning for Molecules and Materials.

Background

I am by no means an expert in this field. Here are books and articles suggest by Taco Cohen when asked references to learn group theory and representation theory.

  1. Carter, Visual Group Theory
    Note: very basic intro to group theory

  2. Theoretical Aspects of Group Equivariant Neural Networks
    Carlos Esteves
    Note: covers all the math you need for equivariant nets in a fairly compact and accessible manner.

  3. Serre, Linear Representations of Finite Groups
    Note: classic text on representations of finite groups. First few chapters are relevant to equivariant nets.

  4. G B Folland. A Course in Abstract Harmonic Analysis
    Note: covers representations of locally compact groups; induced representations.

  5. David Gurarie. Symmetries and Laplacians: Introduction to Harmonic Analysis, Group Representations and Applications.

  6. Mark Hamilton. Mathematical Gauge Theory: With Applications to the Standard Model of Particle Physics
    Note: covers fiber bundles, useful for understanding homogeneous G-CNNs and Gauge CNNs.

TO READ

There are many paper I haven't read carefully yet.

  1. Making Convolutional Networks Shift-Invariant Again
    Richard Zhang ICML 2019 paper
  2. Probabilistic symmetries and invariant neural networks
    Benjamin Bloem-Reddy, Yee Whye Teh JMLR paper
  3. On Representing (Anti)Symmetric Functions
    Marcus Hutter paper
  4. PDE-based Group Equivariant Convolutional Neural Networks
    Bart M.N. Smets, Jim Portegies, Erik J. Bekkers, Remco Duits paper

About

Paper list for equivariant neural network