APMLA-2021 / APMLA-WS_21-22_material

Advanced Probabilistic Machine Learning and Applications (2022)

Winter semester 2021/2022.

Course information

Lecturer: Caterina De Bacco

For any general question about the course, use GitHub issues. Before posting, please make sure your question has not been previously answered. Only in case of private question, send us an email.

Plan: Thu 21st Oct 2021 - Fry 11th Feb 2022, 14 weeks, 2+2 hr/week, 14 weeks, 56hr

NEWS (7.02.2022): If you are not able to register online for the exam (either via Campus / ALMA) it is enough to come directly to the exam. Your grades will then be submitted directly to the University admin after correction and thus your participation confirmed by that. Hence, do not worry if you are not able to register online.

NEWS (10.01.22): Given the current covid-19 situation, we will have ALL the remaining the TUTORIALS online, starting from Friday 14.01.22 (included). The zoom link is here, password: 140122.

NEWS (16.12.21): Assignment has been uploaded and is ready to be tackled! Please use the pseudo-code that you find inside the Assignment folder as a template to write the solutions, rename it as surname_name_A_2022.ipynb (if you are submitting by yourself, otherwise surname1_name1_surname2_name2_A_2022.ipynb if you are submitting in a 2-people group. Similarly, please upload a unique .zip file containing the notebook and in case additional material (e.g. figures), again using the format surname_name_A_2022.zip (and similar for groups of 2 people). The assignement should be submitted at this submission form. Teams can be made of 1 or 2 people.

Submission deadline: Friday 21.01.22 (24.00pm).

NEWS (10.11.21):

1. Exam dates are confirmed:

• Thursday, 10.02.22, 14.00-16.00, room N2 (Klausur).
• Monday, 11.04.22, 13.45-15.45, room A3M03 (instead of D4A19, news: 3.2.22) (Nachklausur).

These are waiting for confirmation of room availability.

2. The time of the Thursday lecture has been anticipated by 15mins, so that we start at 14.00 (instead of 14.15). The class will thus finish at 15.45 (instead of 16.00).

NEWS (12.10.21): the tutorials (Fridays) have been shifted to the 16.15-17.45 time slot (still on Fridays and in person at A301).

NEWS (4.10.21): the theory part of the class (Thursdays) will take place online on zoom. The tutorials (Fridays) instead will take place in presence (see below).

Lectures: Thursdays 14:00-15.45 pm online here, password: 211021.

Tutorials: Fridays 16.00-18.00pm in presence at A301 (Sand). (To be coordinated with the TA for the starting time at 16.00 or 16.15). NEWS (10.01.22): we will have ALL the remaining the TUTORIALS online, starting from Friday 14.01.22 (included). The zoom link is here, password: 140122.

Feedback: after every lecture you are free to give a feedback here.

Lecture-free days: Friday, from Dec 24th 2021 bis Sat Jan 8th 2022 (Weihnachtspause). The lecture on Thursday Dec 23rd will NOT take place.

Prerequisites: all the previous knowledge required by the class Probabilistic Machine Learning applies here too. Nothing extra is needed, so if you are following that class, you are good to go here too. There will be some overlapping contents for sure between the two classes, although my class is more focused on the application side.

Grading : Maximum between 70% written exam+30% assignments and 100% exam.

• Every assignment is composed by several exercises, which will be released sequentially. Information about assignment submission will be provided later in time but it will be made electronically.

• Assignment may be done and submitted in groups of up to 3 people (optional).

Tentative program and schedule

1. Introduction to probabilistic machine learning
   • Reference: Chapter 2 up to Section 2.3.6 and Section 8.2 of Bishop
2. Gaussian Mixture Model (GMM) + Expectation Maximization
   • Reference: Section 9.2 of Bishop
3. Bayesian Mixture Models + Gibbs Sampling
4. Mean Field approach
   • Reference: AMFM
5. TAP approximation
   • Reference: AMFM
6. Bethe Approximation and Belief Propagation part I
   • Reference: MM
7. Bethe Approximation and Belief Propagation part II
8. Stochastic Block Model
9. GMMs + Variational Inference (VI)
10. Poisson matrix factorization
11. Probabilistic matrix factorization for recommender systems
12. VI + LDA
13. Advanced VI: Stochastic VI and Black Box VI

References

• Bishop=C. M. Bishop, Pattern recognition and machine learning (Springer, 2006).
• AMFM=M. Opper and D. Saad, Advanced mean field methods: Theory and practice (MIT press, 2001).
• MM= M. Mèzard and A. Montanari, Information, Physics and Computation (Oxford Graduate texts, 2009).