Fanerst / Papers

Constantly add papers I need or interested

Paper Repository

Papers of my interest

Learning

Learning theory

• Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition [paper]

Generative models

VAEs

• Auto-Encoding Variational Bayes [paper]
• Hierarchical Representations with Poincaré Variational Auto-Encoders [paper]

Autoregressive models

• The Neural Autoregressive Distribution Estimator(NADE) [paper]
• MADE: Masked Autoencoder for Distribution Estimation(MADE) [paper]
• Fast Generation for Convolutional Autoregressive Models [paper]
  Comments: proposed a way to accelerate autoregreesive models like wave net and pixelcnn, need second thought about whether it can be used into VAN
• Autoregressive Quantile Networks for Generative Modeling [paper]
• Transformation Autoregressive Networks [paper]

Flow-based networks

• Normalizing Flow [paper]
• Monge-Ampere Flow [paper]
• Neural Ordinary Differential Equations [paper]
• Density Estimation using Real NVP [paper]
• Improved Variational Inference with Inverse Autoregressive Flow [paper]
• Glow: Generative Flow with Invertible 1×1 Convolutions [paper]

Unclassified

• Variational discriminator bottleneck: improving imitation learning, inverse RL, and GANs by constraining information flow [paper]

Nerual network and MCMC

• Variational Rejection Sampling [paper]
• Discriminator Rejection Sampling [paper]
• Metropolis-Hastings Generative Adversarial Networks [paper]
• Generative Adversarial Training for Markov Chains [paper]
• A-NICE-MC: Adversarial Training for MCMC [paper]
  Comments: improvement of above paper using HMC and NICE to improve MCMC, could it be used to physical system?
• Metropolis-Hastings view on variational inference and adversarial training [paper]

Graph Neural Networks

Reviews

• Graph Neural Networks: A Review of Methods and Applications [paper]
• A Comprehensive Survey on Graph Neural Networks [paper] *to read
• Relational inductive biases, deep learning, and graph networks [paper] *to read
• A Tutorial on Network Embeddings [paper]

Graph embeddings

• GraphGAN: Graph Representation Learning with Generative Adversarial Nets [paper]
• Learning Steady-States of Iterative Algorithms over Graphs [paper]

Graph Convolutional Networks

• Semi-Supervised Classification with Graph Convolutional Networks(GCN) [paper]
• Simplifying Graph Convolutional Networks(SGCN) [paper]
• HOW POWERFUL ARE GRAPH NEURAL NETWORKS?(GRAPH ISOMORPHISM NETWORK) [paper]
• GRAPH WAVELET NEURAL NETWORK [paper]
• Predict then Propagate: Graph Neural Networks meet Personalized PageRank [paper]
• Graph Attention Networks [paper]
• PAN: Path Integral Based Convolution for Deep Graph Neural Networks [paper]

Relational networks

• A simple neural network module for relational reasoning(RN) [paper]
• Recurrent Relational Networks [paper]

GNN for combinatorial optimization problems

• Learning Combinatorial Optimization Algorithms over Graphs [paper]
  Comments: pretty interesting, but don't understand the point for now, probably because of lacking basic knownledge about reinforcement learning.

• Coloring Big Graphs with AlphaGoZero [paper]

GNN adversial attack

• Adversarial Attacks on Neural Networks for Graph Data [paper]
• Can Adversarial Network Attack be Defended? [paper]

Topological interpretation of ML

• Topology of Learning in Artificial Neural Networks [paper]
  Commments: find a way to visualize evolution of weights through training, hinting that the learning ability comes from branching off of weights

• A Geometric View of Optimal Transportation and Generative Model [paper]

Optimal transport at ML

• Latent Space Optimal Transport for Generative Models [paper]
• Wasserstein Dependency Measure for Representation Learning [paper]

Energy Based Model(EBM)

• A Tutorial on Energy-Based Learning [paper]
• Implicit Generation and Generalization in Energy-Based Models [paper]

Gradient Estimators

• Neural Variational Inference and Learning in Belief Networks[paper]
• Reweighted Wake-Sleep [paper]
• Importance Weighted Autoencoders [paper]
• Variational Inference for Monte Carlo Objectives [paper]

Tensor Networks

• Matrix Product Operators for Sequence to Sequence Learning [paper]

Thermodynamics and Statistical Mechanics

• Stochastic Thermodynamic Interpretation of Information Geometry [paper, support material]
• Kinked Entropy and Discontinuous Microcanonical Spontaneous Symmetry Breaking [paper, support material]

Replica symmetry breaking

• Infinite Number of Order Parameters for Spin-Glasses [paper] *to read
• Order Parameter for Spin-Glasses [paper] *to read
• The order parameter for spin glasses: a function on the interval 0-1 [paper] *to read

Message Passing

Belief propagation

• Understanding Belief Propagation and its Generalizations [paper]
• Correctness of belief propagation in Gaussian graphical models of arbitrary topology [paper]

Approximate Inference

ADATAP

• Tractable Approximations for Probabilistic Models: The Adaptive Thouless-Anderson-Palmer Mean Field Approach [paper]
• Adaptive and self-averaging Thouless-Anderson-Palmer mean-field theory for probabilistic modeling [paper]

Expectation propagation

• Expectation Propagation for Approximate Bayesian Inference(EP) [paper]
• Expectation Consistent Approximate Inference(EC) [paper]
• Expectation Consistent Approximate Inference: Generalizations and Convergence(GEC) [paper]
• Expectation Propagation for Exponential Families [paper]

Density consistency

• Loop corrections in spin models through density consistency [paper]

Compressed Sensing

• Message-passing algorithms for compressed sensing(AMP) [paper]
• Vector Approximate Message Passing(VAMP) [paper]
• Compressed Sensing by Shortest-Solution Guided Decimation(SSD) [paper] *to read

Physics and Learning

• Neural Network Renormalization Group [paper, support material]
• Uncover the Black Box of Machine Learning Applied to Quantum Problem by an Introspective Learning Architecture [paper]

Markov Chain Monte Carlo

###MCMC of graphs

• Fastest Mixing Markov Chain on a Graph[paper]

  **Comments**: A way to find optimal transfer matrix of MCMC on graphs. How can it been used in our research, and how to improve it. 

Hamiltonian Monte Carlo

• MCMC Using Hamiltonian Dynamics [paper] *to read
• A Conceptual Introduction to Hamiltonian Monte Carlo [paper] *to read

Network Science

• Estimating network structure from unreliable measurements [paper]
• Spectral redemption in clustering sparse networks [paper]
  Comments: using non-backtracking operator in spectral method to do clustering in sparse networks, proving superiority of non-backtracking operator than other structural matrices like adjacency matrix.
• Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications [paper]

**Comments**: using cavity method to construct belief propagation algorithm to solve stochastic block model, some equations needed to be revisited.

Self

Gravity

• The effects of massive graviton on the equilibrium between the black hole and radiation gas in an isolated box [paper]

To be read

PRL

• Low-Scaling Algorithm for Nudged Elastic Band Calculations Using a Surrogate Machine Learning Model [paper]
• Dynamical Computation of the Density of States and Bayes Factors Using Nonequilibrium Importance Sampling [paper]
• Random Language Model [paper]

Spin Glass and Statistical Physics

• Materials Physics and Spin Glasses [paper]
• Compressed sensing reconstruction using Expectation Propagation [paper]

Cavity method and matrix product state

• A matrix product algorithm for stochastic dynamics on networks, applied to non-equilibrium Glauber dynamics [paper]
• The matrix product approximation for the dynamic cavity method [paper]

Physics and Learning

• Learning a Gauge Symmetry with Neural-Networks [paper]

Machine Learning

Theory

• Theoretical Impediments to Machine Learning With Seven Sparks from the Causal Revolution [paper]
• A lattice-based approach to the expressivity of deep ReLU neural networks [paper]
• Deep learning as closure for irreversible processes: A data-driven generalized Langevin equation [paper]
• On the Impact of the Activation Function on Deep Neural Networks Training [paper]
• MEAN-FIELD ANALYSIS OF BATCH NORMALIZATION [paper]
• A MEAN FIELD THEORY OF BATCH NORMALIZATION [paper]

Generative Model

• Latent Translation: Crossing Modalities by Bridging Generative Models [paper]
• GANSYNTH: ADVERSARIAL NEURAL AUDIO SYNTHESIS [paper]
• Semi-supervised Learning with Deep Generative Models [paper]
• Diagnosing and Enhancing VAE Models [paper]

Structured Data

• The Emerging Field of Signal Processing on Graphs [paper]
• Discriminative Embeddings of Latent Variable Models for Structured Data [paper]

Others

• Hyperbolic Neural Networks [paper]
• Bayesian Neural Networks at Finite Temperature [paper]