IGEL - MP-GNN Benchmark

This repository contains a replica of this repository associated with the paper "Breaking the Limits of Message Passing Graph Neural Networks" published in ICML2021. It extends the provided reproducibility scripts so that:

IGEL is introduced in the graphlet counting (counting.py), isomorphism detection (sr25.py, graph8c.py, and exp_classify.py), and graph classification (mutag.py, ptc.py, proteins.py and enzymes.py).
The results of the original paper can be reproduced with different seeds using Slurm.
The results can be evaluated and fed into our IGEL paper.

For details on IGEL, visit the main repository here.

Original README.

Breaking the Limits of Message Passing Graph Neural Networks

This repository consists of codes of "Breaking the Limits of Message Passing Graph Neural Networks" paper published in ICML2021.

Requirements

These libraries' versions are not strictly needed. But these are the configurations of our test machine. Also, all dependencies of pytorch-geometric are needed.

Python==3.8.5
pytorch==1.5.1
pytorch_geometric==1.6.1
numpy==1.19.1
scipy==1.5.2
networkx==2.5.1
matplotlib==3.3.1

For tensorflow codes our test environment has following libraries' versions

Python==3.6.5
tensorflow-gpu==1.15.0
numpy==1.17.4
matplotlib==3.1.2
scipy==1.3.1
networkx==2.4
pickle==4.0

Reproducing Results

In order to easy debugging and readability, we hardcoded hyperparameters of each model and also selection of model out of MLP, GCN, GAT, GIN, ChebNet, PPGN and our proposals named GNNML1 and GNNML3 in each problem set concerned script file separately. Desired model and its hyper-parameters can be changed according to your selection. Note all provided scripts (except isomorphism test codes) are for single run. To get given statistics in the paper, they should be run many times under different random seeds.

Table-1

graph8c.py, sr25.py, exp_iso.py and exp_classify.py is for each column's of Table1 respectively.

Table-2

counting.py is for result of Table2. For each column, ntask variable in Line 381, can be set out of 0,1,2,3,4 accordingly.

Table-3

For first 3 column of Table3, filtering.py script can be run with desired task by selection of ntask in Line 302 out of 0,1 or 2. Last colmun's result can be taken using freqclass.py script.

Table-4

For Zinc dataset result, Zinc12k.py script is needed for all methods. For Mnist75 result mnist75.py script is needed for all methods except GNML3. For GNNML3, first you should run prepareMnist_gnnml3_tf.py for preprocessing step. It saves the preprocessed data into file. Then mnist75_gnnml3_tf.py should be run.

Table-5

Each column can be obtained by mutag.py, enzymes.py, enzymes_contfeat.py, proteins.py and ptc.py respectively. Just GNNML3 result for 3rd column can also be obtained by tensorflow implementation by using enzymes_contfeats_gnnml3_tf.py

Citation

Please cite this paper if you find it usefull.

@inproceedings{
balcilar2021breaking,
title={Breaking the Limits of Message Passing Graph Neural Networks},
author={Muhammet Balcilar and Pierre H{\'e}roux and Benoit Ga{\"u}z{\`e}re and Pascal Vasseur and S{\'e}bastien Adam and Paul Honeine},
booktitle={Proceedings of the 38th International Conference on Machine Learning (ICML)},
year={2021}	
}

License

MIT License

nur-ag / gnn-matlang