Reading Notes 2017/2018
Author: Robert T. Lange
This repository contains simple reading notes, thoughts, questions and summaries of the papers/book chapters, which I (Robert T. Lange) have read in the second half of 2017 and 2018. This includes the summer break, where I attended a Free-Energy Principle summer school organised by Prof. Blankenburg and Prof. Ostwald (both BCCN and FU Berlin), the DS^3 Summer School as well as the European Summer School in Information Retrieval (ESSIR) and the time of my Computing (ML) Master's at Imperial College London.
The documents are grouped by overarching topic. First of all I hope that this way I am able to structure my knowledge gains and have a quick read to remind myself of keypoints. Second, I hope that you are able to follow my current interests and research progress.
Notes:
- If there is a tick in the box, there exists a summary markdown file in the subdirectory. Otherwise, I have only read the document. Furthermore, I list below the chapters of books, while the markdown file contains the summary for the full (or the parts that I have worked in) book.
- Most papers focus on Deep RL and Hierarchical RL (due to thesis and general interest).
| Read / Notes | Title & Author | Year | Category | Conference | Paper | Notes |
|---|---|---|---|---|---|---|
| 🔥 #13 - 01/20 | Merel et al. - Deep Neuroethology of a Virtual Rodent | 2020 | DRL-Neuro | ICLR | Click | Click |
| 🔥 #12 - 12/19 | Gaier & Ha - Weight Agnostic Neural Networks | 2019 | NAS | NeuRIPS | Click | Click |
| 🔥 #11 - 11/19 | Kümmerer et al. - Saliency Benchmarking made easy: Separating models, maps and metrics | 2018 | Saliency | EECV | Click | Click |
| 🔥 #10 - 08/19 | Baydin et al. - Automatic Differentiation in Machine Learning: a Survey | 2018 | Autodiff | JMLR | Click | Click |
| 🔥 #9 - 08/19 | Flennerhag et al. - Transferring Knowledge across Learning Processes | 2019 | Meta-Learning | ICLR | Click | Click |
| 🔥 #8 - 08/19 | Jacot et al. - Neural Tangent Kernel: Convergence and Generalization in Neural Networks | 2018 | Theory of DL | NeuRIPS | Click | Click |
| 🔥 #7 - 08/19 | Collins et al. - Capacity and Trainability in Recurrent Neural Networks | 2017 | RNNs | ICLR | Click | Click |
| 🔥 #6 - 08/19 | Li et al. - A Generalized Framework for Population Based Training | 2019 | PBT | ArXiv | Click | Click |
| 🔥 #5 - 08/19 | Jaderberg et al. - Population Based Training of Neural Networks | 2017 | PBT | ArXiv | Click | Click |
| 🔥 #4 - 08/19 | Frankle et al. - Stabilizing The Lottery Ticket Hypothesis | 2019 | Initialization | ArXiv | Click | Click |
| 🔥 #3 - 08/19 | Frankle & Carbin - The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks | 2019 | Initialization | ICLR | Click | Click |
| 🔥 #2 - 08/19 | Nayebi et al. - Task-Driven Convolutional Recurrent Models of the Visual System | 2018 | RNNs | NeuRIPS | Click | Click |
| 🔥 #1 - 07/19 | Bengio et al. - A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms | 2019 | Meta | ArXiv | Click | Click |
2019-03
- Finn et al (2018) - Probabilistic Model-Agnostic Meta-Learning
- Andrychowicz et al (2016) - Learning to learn by gradient descent by gradient descent
- Finn et al (2017) - Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Multi-Agent RL
2018-01
- Das et al (2019) - TarMAC: Targeted Multi-Agent Communication
- Hausknecht and Stone (2015) - Deep Recurrent Q-Learning for POMDPs
2018-11
- Strouse et al (2018) - Learning to Share and Hide Intentions using Info Regularization
- Foerster et al (2016) - Learning to Communicate with Deep MARL
Biologically-Plausible Deep Learning
2019-02
- Whittington & Bogacz (2019) - Theories of Error Propagation in the Brain
2018-12
- Sacramento et al (2018) - Dendritic cortical microcircuits approximate the backpropagation algorithm
- Bartunov et al (2018) - Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures
- Lillicrap et al (2016) - Random synaptic feedback weights support error backpropagation for deep learning
2018-11
- Garnelo et al (2018b) - Neural Processes
Hierarchical Reinforcement Learning
2018-08
- Pastra and Aloimonos (2012) - The minimalist grammar of action
- Bacon et al (2017) - The Option-Critic Architecture
- Daniel et al (2016) - Probabilistic inference for determining options in reinforcement learning
- Smith et al (2018) - An Inference-Based Policy Gradient Method for Learning Options
2018-07
- McGovern & Sutton (1998) - Macro-Actions in Reinforcement Learning: An Empiricial Analysis
- McGovern et al (1997) - Roles of Macro-Actions in Accelerating Reinforcement Learning
2018-06
- Yao et al (2014) - Universal Option Models
- Levy et al (2018) - Hierarchical Reinforcement Learning with Hindsight
- Bakker and Schmidhuber (2004) - Hierarchical Reinforcement Learning Based on Subgoal Discovery and Subpolicy Specialization
- Mannor et al (2004) - Dynamic Abstraction in Reinforcement Learning via Clustering
- Menache et al (2002) - Q-Cut - Dynamic Discovery of Sub-Goals in Reinforcement Learning
- Stolle and Barto (2002) - Learning Options in Reinforcement Learning
- McGovern and Barto (2001) - Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density
2018-05
- Frans et al (2018) - Meta Learning Shared Hierarchies
- Florensa et al (2017) - Stochastic Neural Networks for Hierarchical Reinforcement Learning
Formal Grammars, Grammatical Inference and Surprisal
2018-07
- Siyari et al (2016) - Lexis: An Optimization Framework
2018-06
- Stout et al (2018) - Grammars of action in human behavior and evolution
2018-05
- Hale (2014) - Automaton theories of human sentence comprehension
- Schoenhense & Faisal (2017) - Data-efficient inference of hierarchical structure in sequential data by information-greedy grammar inference
- Hale (2001) - A Probabilistic Earley Parser as a Psycholinguistic Model
(Deep) Reinforcement Learning
2018-06
- Schaul et al (2015) - Universal Value Function Approximators
- Gershman and Daw (2017) - Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework
- Rusu et al (2016) - Policy Distillation
- Andrychowicz et al (2018) - Hindsight Experience Replay
- Choshen, Fox, Loewenstein (2018) - DORA - Directed Outreaching Reinforcement Action-Selection
2018-05
- Li (2017) - Deep Reinforcement Learning: An Overview
- Arulkumaran (2017) - A Brief Survey of Deep Reinforcement Learning
- Dayan (1993b) - Improving Generalisation for Temporal Difference Learning: The Successor Representation
2017-08
- Barto & Sutton (2016 - draft) - Ch. 1: The Reinforcement Learning Problem
Free-Energy Principle
2017-07
- Friston (2010) - The free-energy principle: a unified brain theory?
- Limanowski, Blankenburg (2013) - Minimal self-models and the free energy principle
2017-08
- Ostwald (2015) - The Free Energy Principle for Perception: An Introduction
- Bogacz (2017) - A tutorial on the free-energy framework for modelling perception and learning
Variational Inference
2019-02
- Zhang et al. (2018) - Advances in Variational Inference
2017-07
- Ostwald et al. (2014) - A tutorial on variational Bayes for latent linear stochastic time-series models
Deep Learning (Application + Theory)
2019-06
- Morcos et al (2018) - Insights on representational similarity in neural networks with canonical correlation
2019-03
- Spoerer et al (2017) - Recurrent Convolutional NNs: A Better Model of Biological Object Recognition
2017-07
- Goodfellow et al. (2016) - Ch. 6: Deep Feedforward Networks
- Gal, Ghahramani (2015) - Dropout as a Bayesian Approximation: Insights and Applications
2017-08
- Gal & Ghahramani (2016) - On Modern Deep Learning and Variational Inference
Optimization
2017-07
- Ruder (2016) - An overview of gradient descent optimization algorithms
Bayesian Optimization & Gaussian Processes
2017-07
- Shahriari et al. (2015) - Taking the Human Out of the Loop: A Review of Bayesian Optimization
Real Intelligence
2017-08
- Jeff Hawkins (2003) - On Intelligence
- Hassabis (2017) - Neuroscience-Inspired Artificial Intelligence
Machine Learning Reading Group ICL
2017-10
- #1: Daume (2004) - From Zero to Reproducing Kernel Hilbert Spaces in Twelve Pages or Less
- #2: Zhang et al (2017) - Theory of Deep Learning III: Generalization Properties of SGD
- #3: Marco et al (2017) - Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization
2017-11
- #5: Lee et al (2017) - Deep Neural Networks as Gaussian Processes