🍀 clover 🍀

Pytorch code for "Understanding and Improving Features Learned in Deep Functional Maps" - CVPR 2023

TLDR: To improve non-rigid shape matching experiments in the deep functional map setting, we recommend enforcing properness (see § 4.1) using DiffusionNet. Alternatively, incorporate smoothing blocks after each layer in your preferred architectures.

👷 Installation

This implementation requires Python >= 3.7. Install dependencies using pip:

pip install -r requirements.txt

📖 Usage

While this paper does not introduce a new architecture per se, it does provide improvements to existing network architectures within the deep functional maps framework. This repository includes code for producing the network architecture we used, in addition to how to compute a proper functional map.

These code blocks can be integrated into your preferred Deep Functional Maps codebase.

🥼 ASAP Network Architectures

In the paper, we evaluated three network architectures: DiffusionNet, DGCNN, and DeltaConv. For DiffusionNet, we used the original code base provided by the authors. For DGCNN and DeltaConv, we modified the original code base to include smoothing blocks, as described in § 4.2 of the paper. All code for these network architectures is located in the networks folder.

Each network takes an input feature (such as xyz coordinates, HKS, etc.) of size n x f, where n is the number of points and f is the number of features. The output of the network is a feature of size n x d, where d is the dimension of the output space. In addition, they take the spectral information of the shape (eigenvalues and eigenvectors of the Laplacian, and the mass vector) as input. For DiffusionNet, the tensor of gradients is also used, while DeltaConv additionally uses normals as per the original implementation.

〽️ Enforcing Properness

We provide the code for enforcing properness of the functional map in fmap_utils.py. The function proper_fmap_computation takes as input the functional map and the spectral information of the shapes, along with the features used to compute the fmaps, and returns the proper functional map.

We recommend testing both the original and the proper version of the functional map in the loss function, depending on the task and dataset, to determine which combination yields the best results.

📈 Results

If you wish to report our result, we have summarized them below. Our method is referred to as GeomFmaps - clover. X on Y indicates that the method was trained on dataset X and tested on dataset Y.

Near Isometric Shape Matching: We provide results on the FAUST (F), Scape (S), and SHREC (SH) datasets. We report the mean geodesic error, following the protocol used in all deep functional map papers, in both supervised and unsupervised settings.

Method	F on F	S on S	F on S	S on F	F on SH	S on SH
GeomFmaps - clover (supervised)	2.6	2.9	3.4	2.7	5.7	5.0
GeomFmaps - clover (unsupervised)	3.3	3.9	4.2	3.3	6.2	5.3

Non-Isometric Shape Matching: We provide results on the SMAL dataset. We report the mean geodesic error, following the same protocol as in all the deep functional maps papers, in both the supervised and unsupervised settings.

Method	SMAL
GeomFmaps - clover (supervised)	4.7
GeomFmaps - clover (unsupervised)	5.1

🎓 Citation

If you find this work useful in your research, please consider citing:

@inproceedings{attaiki2023clover,
    title={Understanding and Improving Features Learned in Deep Functional Maps},
    author={Souhaib Attaiki and Maks Ovsjanikov},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    month={June},
    year={2023}
}

pvnieo / clover