Awesome Sum-Product Networks

awesome-spn is a curated and structured list of resources about Sum-Product Networks (SPNs), tractable deep density estimators.

This includes (even not formally published) research papers sorted by year and topics as well as links to tutorials and code and other related Tractable Probabilistic Models (TPMs). It is inspired by the SPN page at the Washington University.

Licence and Contributing

awesome-spn is released under Public Domain. Feel free to complete and/or correct any of these lists. Pull requests are very welcome!

Papers
- Year
  - 2017
  - 2016
  - 2015
  - 2014
  - 2013
  - 2012
  - 2011
- Topic
Related Works
Resources
References

Papers

Sorted by year or topics

Year

2012

2011

[Delalleau2011] Shallow vs. Deep Sum-Product Networks NIPS2011 theory
[Poon2011] Sum-Product Networks: A New Deep Architecture UAI2011 modeling weight-learning

Topics

Weight Learning

[Trapp2017] Safe Semi-Supervised Learning of Sum-Product Networks semi supervised
[Zhao2017] Efficient Computation of Moments in Sum-Product Networks ADF
[Jaini2016] Online Algorithms for Sum-Product Networks with Continuous Variables OBMM
[Desana2016] Learning Arbitrary Sum-Product Network Leaves with Expectation-Maximization EM
[Zhao2016b] A unified approach for learning the parameters of sum-product networks CCCP
[Zhao2016a] Collapsed Variational Inference for Sum-Product Networks variational method
[Rashwan2016] Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks OBMM EGD
[Peharz2014b] Learning Selective Sum-Product Networks ML SSPN
[Poon2011] Sum-Product Networks: A New Deep Architecture EM Hard EM SGD
[Gens2012] Discriminative Learning of Sum-Product Networks disc Hard EM disc Hard SGD

Structure Learning

[DiMauro2017] Alternative Variable Splitting Methods to Learn Sum-Product Networks RGVS EBVS
[Hsu2017] Online Structure Learning for Sum-Product Networks with Gaussian Leaves ICLR 2017 - Workshop online structure learning
[Trapp2016] Structure Inference in Sum-Product Networks using Infinite Sum-Product Trees infiniteSPT Bayesian nonparametrics
[Melibari2016c] Dynamic Sum-Product Networks for Tractable Inference on Sequence Data hill-climbing
[Rahman2016] Merging Strategies for Sum-Product Networks: From Trees to Graphs pruning dagSPN
[Vergari2015] Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning LearnSPN-b LearnSPN-bt LearnSPN-btb
[Dennis2015] Greedy Structure Search for Sum-Product Networks dagSPN
[Adel2015] Learning the Structure of Sum-Product Networks via an SVD-based Algorithm SPN-SVD DSPN-SVD
[Nath2015] Learning Relational Sum-Product Networks relational
[Lee2014] Non-Parametric Bayesian Sum-Product Networks non-parametrics
[Peharz2014b] Learning Selective Sum-Product Networks SSPN
[Rooshenas2014] Learning Sum-Product Networks with Direct and Indirect Interactions ID-SPN
[Lee2013] Online Incremental Structure Learning of Sum-Product Networks
[Peharz2013] Greedy Part-Wise Learning of Sum-Product Networks bottom-up
[Gens2013] Learning the Structure of Sum-Product Networks top-down LearnSPN
[Dennis2012] Learning the Architecture of Sum-Product Networks Using Clustering on Variables top-down``k-means

Representation Learning

[Vergari2017] Encoding and Decoding Representations with Sum- and Max-Product Networks decoding
[Vergari2016] Visualizing and Understanding Sum-Product Networks embeddings

Modeling

[Sharir2017] Sum-Product-Quotient Networks SPQN
[Zheng2017] Learning Semantic Maps with Topological Spatial Relations Using Graph-Structured Sum-Product Networks GraphSPN
[Desana2017] Sum-Product Graphical Models SPGM
[Mauà2017] Credal Sum-Product Networks CSPN
[Gens2017] Compositional Kernel Machines CKM
[Friesen2017] Unifying Sum-Product Networks and Submodular Fields SSPN
[Molina2017] Poisson Sum-Product Networks: A Deep Architecture for Tractable Multivariate Poisson Distributions Poisson SPNs
[Melibari2016c] Dynamic Sum-Product Networks for Tractable Inference on Sequence Data dynamic-SPN
[Melibari2016b] Sum-Product-Max Networks for Tractable Decision Making decision-diagram
[Melibari2016a] Decision Sum-Product-Max Networks decision-diagram
[Friesen2015] Recursive Decomposition for Nonconvex Optimization opt
[Niepert2015] Learning and Inference in Tractable Probabilistic Knowledge Bases relational
[Nath2015] Learning Relational Sum-Product Networks relational
[Nath2014] Learning Tractable Statistical Relational Models relational
[Peharz2014b] Learning Selective Sum-Product Networks SSPN
[Stuhlmueller2012] Dynamic Programming Algorithm for Inference in Recursive Probabilistic Programs FSPN
[Poon2011] Sum-Product Networks: A New Deep Architecture SPN

Applications

[Pronobis2017a] Deep Spatial Affordance Hierarchy: Spatial Knowledge Representation for Planning in Large-scale Environments SSRR 2017 robot control
[Rathke2017] Locally Adaptive Probabilistic Models for Global Segmentation of Pathological OCT Scans MICCAI 2017 segmentation
[Friesen2017] Submodular Sum-Product Networks for Scene Understanding OpenReview@ICLR 2017 segmentation
[Sguerra2016] Image Classification Using Sum-Product Networks for Autonomous Flight of Micro Aerial Vehicles image-classification ID-Spn
[Yuan2016] Modeling Spatial Layout for Scene Image Understanding Via a Novel Multiscale Sum-Product Network cv segmentation
[Nath2016] Learning Tractable Probabilistic Models for Fault Localization
[Wang2015] Hierarchical Spatial Sum-Product Networks for action recognition in Still Images cv activity-recognition
[Amer2015] Sum Product Networks for Activity Recognition cv activity-recognition
[Li2015] Combining Sum-Product Network and Noisy-OrModel for Ontology Matching sem-web
[Cheng2014] Language Modeling with Sum-Product Networks sequence
[Ratajczak2014] Sum-Product Networks for Structured Prediction: Context-Specific Deep Conditional Random Fields speech
[Peharz2014a] Modeling Speech with Sum-Product Networks: Application to Bandwidth Extension speech
[Amer2012] Sum-product Networks for Modeling Activities with Stochastic Structure cv``activity-recognition

Theory

[Mei2017] Maximum A Posteriori Inference in Sum-Product Networks MAP inference
[Conaty2017] Approximation Complexity of Maximum A Posteriori Inference in Sum-Product Networks MAP inference
[Zhao2016b] A Unified Approach for Learning the Parameters of Sum-Product Networks CCCP
[Peharz2016] On the Latent Variable Interpretation in Sum-Product Networks EM
[Friesen2016] The Sum-Product Theorem: A Foundation for Learning Tractable Models opt sum-prod-theorem
[Peharz2015b] Foundations of Sum-Product Networks for Probabilistic Modeling
[Friesen2015] Recursive Decomposition for Nonconvex Optimization opt sum-prod-theorem
[Zhao2015] On the Relationship between Sum-Product Networks and Bayesian Networks
[Peharz2015a] On Theoretical Properties of Sum-Product Networks
[Martens2014] On the Expressive Efficiency of Sum Product Networks depth
[Delalleau2011] Shallow vs. Deep Sum-Product Networks depth

Related Works

Arithmetic Circuits

[Darwiche2003] [A Differential Approach to Inference in Bayesian Networks](Advances in Neural Information Processing Systems 2011) J. ACM 2003
[Lowd2013] Learning Markov Networks With Arithmetic Circuits AISTATS 2013
[Rooshenas2016] Discriminative Structure Learning of Arithmetic Circuits AISTATS 2016
[Choi2017] On Relaxing Determinism in Arithmetic Circuits ICML 2017

Other TPMs

[Livni2013] A Provably Efficient Algorithm for Training Deep Networks arXiv 2013

Exploiting Sum-Product Theorem

[Gens2017] Compositional Kernel Machines ICLR 2017 - Workshop

Resources

Dataset

20 commonly used datasets for density estimation as in [Lowd2013][Gens2013][Rooshenas2014][Vergari2015][Adel2015][Zhao2016a][Rooshenas2016]

Code

[Desana2017] SPGM implementation in C++
[Pronobis2017b] LibSPN tensorflow implementation with bindings in python3
SumProductNetworks.jl Software package for SPNs. julia
[Hsu2017] Tachyon structure and parameter learning in python3
[Hsu2017] Online structure learning for continuous leaf SPNs python3
[Zhao2016a, Zhao2016b] Parameter optimization using MLE and Bayesian approach spn-opt C++
[Vergari2016] spyn-repr extracting embeddings from SPNs python3
[Vergari2015] spyn LearnSPN-B/T/B and SPN inference routines in Python python3
[Rooshenas2014] ID-SPN and inference routines on ACs implemented in the Libra Toolkit Ocaml
[Peharz2014a] ABE-SPN Artificial Bandwidth-Extension with Sum-Product Networks MATLAB C++
GoSPN implementing LearnSPN in Go Go
[Cheng2014] lmspn Language modeling with SPNs C++ CUDA
C++/Cuda porting of Poon's architecture C++ CUDA
Python porting of Poon's architecture python2
[Gens2013] LearnSPN Java
[Poon2011] Code to train Poon's architecture weigths by EM Java MPI

Talks and Tutorials

Di Mauro and Vergari Learning Sum-Product Networks tutorial at PGM'16 2016
Poupart P. Deep Learning, Sum-Product Networks Part I Part II 2015
Hernàndez-Lobato, J. M. An Introduction to Sum-Product Networks 2013
Gens, R. Learning the Structure of Sum-Product Networks [Gens2013] 2013
Poon, H. Sum-Product Networks: A New Deep Architecture [Poon2011] 2011

References

[Adel2015]
Adel, Tameem and Balduzzi, David and Ghodsi, Ali
Learning the Structure of Sum-Product Networks via an SVD-based Algorithm
Uncertainty in Artificial Intelligence 2015

[Amer2012]
Amer, Mohamed and Todorovic, Sinisa
Sum-Product Networks for Modeling Activities with Stochastic Structure
2012 IEEE Conference on CVPR

[Amer2015]
_Amer, Mohamed and Todorovic, Sinisa_
**Sum Product Networks for Activity Recognition**
IEEE Transactions on Pattern Analysis and Machine Intelligence

[Cheng2014]
_Cheng, Wei-Chen and Kok, Stanley and Pham, Hoai Vu and Chieu, Hai Leong and Chai, Kian Ming Adam_
**Language modeling with Sum-Product Networks**
INTERSPEECH 2014

[Choi2017]
_Cheng, Arthur and Darwiche, Adnan_
**On Relaxing Determinism in Arithmetic Circuits**
ICML 2017

[Conaty2017]
_Conaty, Diarmaid and Deratani Mauá, Denis and de Campos, Cassio P._
**Approximation Complexity of Maximum A Posteriori Inference in Sum-Product Networks**
UAI 2017

[Darwiche2003]
_Darwiche, Adnan_
**A Differential Approach to Inference in Bayesian Networks**
Journal of the ACM 2003.

[Dellaleau2011]
_Delalleau, Olivier and Bengio, Yoshua_
**Shallow vs. Deep Sum-Product Networks**
Advances in Neural Information Processing Systems 2011.

[Dennis2012]
_Dennis, Aaron and Ventura, Dan_
**Learning the Architecture of Sum-Product Networks Using Clustering on Varibles**
Advances in Neural Information Processing Systems 25

[Dennis2015]
_Dennis, Aaron and Ventura, Dan_
**Greedy Structure Search for Sum-product Networks**
International Joint Conference on Artificial Intelligence 2015

[Desana2016]
_Desana, Mattia and Schn{\"{o}}rr Christoph_
**Learning Arbitrary Sum-Product Network Leaves with Expectation-Maximization**
arxiv.org/abs/1604.07243

[Desana2017]
_Desana, Mattia and Schn{\"{o}}rr Christoph_
**Sum-Product Graphical Models**
arxiv.org/abs/1708.06438

[DiMauro2017]
_Di Mauro, Nicola and Esposito, Floriana and Ventola, Fabrizio Giuseppe and Vergari, Antonio_
**Alternative variable splitting methods to learn Sum-Product Networks**
Proceedings of the 16th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2017)

[Friesen2015]
_Friesen, Abram L. and Domingos, Pedro_
**Recursive Decomposition for Nonconvex Optimization**
Proceedings of the 24th International Joint Conference on Artificial Intelligence

[Friesen2016]
_Friesen, Abram L. and Domingos, Pedro_
**The Sum-Product Theorem: A Foundation for Learning Tractable Models**
ICML 2016

[Friesen2017]
_Friesen, Abram L. and Domingos, Pedro_
**Unifying Sum-Product Networks and Submodular Fields**
Principled Approaches to Deep Learning Workshop at ICML 2017

[Gens2012]
_Gens, Robert and Domingos, Pedro_
**Discriminative Learning of Sum-Product Networks**
NIPS 2012

[Gens2013]
_Gens, Robert and Domingos, Pedro_
**Learning the Structure of Sum-Product Networks**
ICML 2013

[Gens2017]
_Gens, Robert and Domingos, Pedro_
**Compositional Kernel Machines**
ICLR 2017 - Workshop Track

[Hsu2017]
_Hsu, Wilson and Kalra, Agastya and Poupart, Pascal_
**Online Structure Learning for Sum-Product Networks with Gaussian Leaves**
ICLR 2017 - Workshop Track

[Jaini2016]
_Jaini, Priyank and Rashwan, Abdullah and Zhao, Han and Liu, Yue and Banijamali, Ershad and Chen, Zhitang and Poupart, Pascal_
**Online Algorithms for Sum-Product Networks with Continuous Variables**
International Conference on Probabilistic Graphical Models 2016

[Krakovna2016]
_Krakovna, Viktoriya and Looks, Moshe_
**A Minimalistic Approach to Sum-Product Network Learning for Real Applications**
ICLR 2016

[Lee2013]
_Lee, Sang-Woo and Heo, Min-Oh and Zhang, Byoung-Tak_
**Online Incremental Structure Learning of Sum-Product Networks**
ICONIP 2013

[Lee2014]
_Lee, Sang-Woo and Watkins, Christopher and Zhang, Byoung-Tak_
**Non-Parametric Bayesian Sum-Product Networks**
Workshop on Learning Tractable Probabilistic Models 2014

[Li2015]
_Weizhuo Li_
**Combining sum-product network and noisy-or model for ontology matching**
Proceedings of the 10th International Workshop on Ontology Matching

[Livni2013]
_Livni, Roi and Shalev-Shwartz, Shai and Shamir, Ohad_
**A Provably Efficient Algorithm for Training Deep Networks**
arXiv 2013

[Lowd2013]
_Lowd, Daniel and Rooshenas, Amirmohammad_
**Learning Markov Networks With Arithmetic Circuits**
Proceedings of the 16th International Conference on Artificial Intelligence and Statistics 2013

[Martens2014]
_Martens, James and Medabalimi, Venkatesh_
**On the Expressive Efficiency of Sum Product Networks**
arXiv/1411.7717

[Mauà2017]
_Mauá, Deratani Denis and Cozman Fabio Gagliardi and Conaty, Diarmaid and de Campos, Cassio P._
**Credal Sum-Product Networks**
ISIPTA 2017

[Mei2017]
_Mei, Jun and Jiang, Yong and Tu, Kewei_
**Maximum A Posteriori Inference in Sum-Product Networks**
arXiv 2017

- [[Mei2017](#mei2017)] [**Maximum A Posteriori Inference in Sum-Product Networks**](https://arxiv.org/abs/1708.04846) *arXiv* [`theory`](#theory) *

[Melibari2016a]
_Melibari, Mazen and Poupart, Pascal and Doshi, Prashant_
**Decision Sum-Product-Max Networks**
Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI 2016)

[Melibari2016b]
_Melibari, Mazen and Poupart, Pascal and Doshi, Prashant_
**Sum-Product-Max Networks for Tractable Decision Making**
Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems

[Melibari2016c]
_Melibari, Mazen and Poupart, Pascal and Doshi, Prashant and Trimponias, George_
**Dynamic Sum-Product Networks for Tractable Inference on Sequence Data**
International Conference on Probabilistic Graphical Models 2016

[Molina2017]
_Molina, Alejandro and Natarajan, Sriraam and Kersting, Kristian_
**Poisson Sum-Product Networks: A Deep Architecture for Tractable Multivariate Poisson Distributions**
Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI 2017)

[Nath2014]
_Nath, Aniruddh and Domingos, Pedro_
**Learning Tractable Statistical Relational Models**
Workshop on Learning Tractable Probabilistic Models

[Nath2015]
_Nath, Aniruddh and Domingos, Pedro_
**Learning Relational Sum-Product Networks**
Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI 2015)

[Nath2016]
_Nath, Aniruddh and Domingos, Pedro_
**Learning Tractable Probabilistic Models for Fault Localization**
Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI 2016)

[Niepert2015]
_Niepert, Mathias and Domingos, Pedro_
**Learning and Inference in Tractable Probabilistic Knowledge Bases**
UAI 2015

[Peharz2013]
_Peharz, Robert and Geiger, Bernhard and Pernkopf, Franz_
**Greedy Part-Wise Learning of Sum-Product Networks**
ECML-PKDD 2013

[Peharz2014a]
_Peharz, Robert and Kapeller, Georg and Mowlaee, Pejman and Pernkopf, Franz_
**Modeling Speech with Sum-Product Networks: Application to Bandwidth Extension**
ICASSP2014

[Peharz2014b]
_Robert Peharz and Gens, Robert and Domingos, Pedro_
**Learning Selective Sum-Product Networks**
Workshop on Learning Tractable Probabilistic Models 2014

[Peharz2015a]
_Robert Peharz and Tschiatschek, Sebastian and Pernkopf, Franz and Domingos, Pedro_
**On Theoretical Properties of Sum-Product Networks**
Proceedings of the 18th International Conference on Artificial Intelligence and Statistics

[Peharz2015b]
_Peharz, Robert_
**Foundations of Sum-Product Networks for Probabilistic Modeling**
PhD Thesis

[Peharz2016]
_Robert Peharz and Robert Gens and Franz Pernkopf and Pedro Domingos_
**On the Latent Variable Interpretation in Sum-Product Networks**
arxiv.org/abs/1601.06180

[Poon2011]
_Poon, Hoifung and Domingos, Pedro_
**Sum-Product Network: a New Deep Architecture**
UAI 2011

[Pronobis2017a]
_Pronobis, A. and Riccio, F. and Rao, R.~P.~N._
**Deep Spatial Affordance Hierarchy: Spatial Knowledge Representation for Planning in Large-scale Environments**
SSRR 2017

[Pronobis2017b]
_Pronobis, A. and Ranganath, A. and Rao, R.~P.~N._
**LibSPN: A Library for Learning and Inference with Sum-Product Networks and TensorFlow**
Principled Approaches to Deep Learning Workshop at ICML 2017

[Rahman2016]
_Tahrima Rahman and Vibhav Gogate_
**Merging Strategies for Sum-Product Networks: From Trees to Graphs**
UAI 2016

**Learning Deep Generative Spatial Models for Mobile Robots**
arXiv 2016

[Rahman2016]
_Tahrima Rahman and Vibhav Gogate_
**Merging Strategies for Sum-Product Networks: From Trees to Graphs**
UAI 2016

[Rashwan2016]
_Rashwan, Abdullah and Zhao, Han and Poupart, Pascal_
**Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks**
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics

[Ratajczak2014]
_Ratajczak, Martin and Tschiatschek, S and Pernkopf, F_
**Sum-Product Networks for Structured Prediction: Context-Specific Deep Conditional Random Fields**
Workshop on Learning Tractable Probabilistic Models 2014

[Rathke2017]
_Rathke, F.; Desana, M. and Schnörr, C._
**Locally Adaptive Probabilistic Models for Global Segmentation of Pathological OCT Scans**
MICCAI 2017

[Rooshenas2014]
_Rooshenas, Amirmohammad and Lowd, Daniel_
**Learning Sum-Product Networks with Direct and Indirect Variable Interactions**
ICML 2014

[Rooshenas2016]
_Rooshenas, Amirmohammad and Lowd, Daniel_
**Discriminative Structure Learning of Arithmetic Circuits**
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics

[Sharir2017]
_Sharir, Or and Shashua, Amnon_
** Sum-Product-Quotient Networks**
arXiv *

[Sguerra2016]
_Sguerra, Bruno Massoni and Cozman, Fabio G._
**Image Classification Using Sum-Product Networks for Autonomous Flight of Micro Aerial Vehicles**
BRACIS 2016 - 5th Brazilian Conference on Intelligent Systems *

[Stuhlmueller2012]
_Stuhlmuller, Andreas and Goodman, Noah D._
**A Dynamic Programming Algorithm for Inference in Recursive Probabilistic Programs**
StaRAI 2012

[Trapp2016]
_Trapp, Martin and Peharz, Robert and Skowron, Marcin and Madl, Tamas and Pernkopf, Franz and Trappl, Robert_
**Structure Inference in Sum-Product Networks using Infinite Sum-Product Trees**
Workshop on Practical Bayesian Nonparametrics at NIPS 2016

[Trapp2017]
_Trapp, Martin and Madl, Tamas and Peharz, Robert and Pernkopf, Franz and Trappl, Robert_
**Safe Semi-Supervised Learning of Sum-Product Networks**
UAI 2017

[Vergari2015]
_Vergari, Antonio and Di Mauro, Nicola and Esposito, Floriana_
**Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning**
ECML-PKDD 2015

[Vergari2016]
_Vergari, Antonio and Di Mauro, Nicola and Esposito, Floriana_
**Visualizing and Understanding Sum-Product Networks**
arXiv:1608.08266

[Vergari2017]
_Vergari, Antonio and Peharz, Robert and Di Mauro, Nicola and Esposito, Floriana_
**Encoding and Decoding Representations with Sum- and Max-Product Networks**
ICLR 2017 - Workshop Track

[Wang2015]
_Wang, Jinghua and Wang, Gang_
**Hierarchical Spatial Sum-Product Networks for action recognition in Still Images**
arXiv:1511.05292

[Yuan2016]
_Zehuan Yuan and Hao Wang and Limin Wang and Tong Lu and Shivakumara Palaiahnakote and Chew Lim Tan_
**Modeling Spatial Layout for Scene Image Understanding Via a Novel Multiscale Sum-Product Network**
Expert Systems with Applications

[Zhao2015]
_Zhao, Han and Melibari, Mazen and Poupart, Pascal_
**On the Relationship between Sum-Product Networks and Bayesian Networks**
ICML 2015

[Zhao2016a]
_Zhao, Han and Adel, Tameem and Gordon, Geoff and Amos, Brandon_
**Collapsed Variational Inference for Sum-Product Networks**
ICML 2016

[Zhao2016b]
_Zhao, Han and Poupart, Pascal and Gordon, Geoff_
**A Unified Approach for Learning the Parameters of Sum-Product Networks**
NIPS 2016

[Zhao2017]
_Zhao, Han and Gordon, Geoff and Poupart, Pascal_
**Efficient Computation of Moments in Sum-Product Networks**
NIPS 2017

[Zheng2017]
_Zheng, Kaiyu and Pronobis, Andrzej and Rao, Rajesh P.N.
**Learning Semantic Maps with Topological Spatial Relations Using Graph-Structured Sum-Product Networks**
arXiv

Tavpritesh / awesome-spn