Probabilistic Machine Learning
CS772A
2017-18 (odd semester)
Instructor: Piyush Rai: (office: RM-502, email: piyush AT cse DOT iitk DOT ac DOT in)
Office Hours: Wed 11:00-12:00 (or by appointment)
Q/A Forum: Piazza
Class Venue: RM-101
Class Timings: Tue/Th 6:00-7:15pm
TAs: Gundeep Arora, Smrithi Prabhu, Vinay Verma, Prem Raj
TA Office Hours: Gundeep (Fri, 3-4pm, RM 515), Smrithi (Mon 3-4pm, RM 515), Vinay (Tue 4-5pm, RM 504)

Pre-requisites

CS771 (Intro to Machine Learning) or equivalent. This requirement can be waived if you have a significant prior exposure to machine learning through course-work or substantial project-work. Note that this course will make extensive use of probability, statistics, and optimization. A solid background on these topics, as well as introductory machine learning, will be essential. Students are also supposed to be familiar with programming in MATLAB/Python.

Grading

(Tentative break-up) There will be 5 pen-and-paper assignments (total 30%), a mid-term (20%), a final-exam (25%), and a course-project (25%). A bonus of up to 5% will be reserved for class/Piazza participation.

Schedule (Tentative)

(*) Re-scheduled class
Date Topics Readings/References Slides/Notes
Aug 1 Course Logistics and Introduction Nature article, Probability Refresher slides slides (print version)
Foundations and Probabilistic Supervised Learning
Aug 3 Basics of Parameter Estimation in Probabilistic Models Parameter Estimation (only up to Section 3.1 for now) slides (print version)
Aug 5 (*) Probabilistic Models for Regression MLAPP Section 7.1-7.3, 7.5.1, 7.6 (up to 7.6.2) slides (print version)
Aug 17 Probabilistic Models for Classification (I): Generative Classification Optional Readings: PRML Section 4.2, MLAPP Section 4.1-4.2.5 slides (print version)
Aug 19 (*) Probabilistic Models for Classification (II): Discriminative Classification Optional Readings: PRML Section 4.3, MLAPP Sections 8.1-8.4, 8.6 slides (print version)
Aug 22 Exponential Family and Generalized Linear Models MLAPP Sections 9.1-9.3, Exponential Family and GLMs slides (print version)
Aug 24 Hyperparameter Estimation in Probabilistic Models PRML Section 3.5 slides (print version)
Aug 26 (*) Working with Gaussians, Linear Gaussian Models MLAPP Sec. 4.1, 4.3-4.4, PRML Sec. 2.3 slides (print version)
Simple Latent Variable Models
Aug 29 Introduction to Latent Variable Models, LVMs for Clustering MLAPP Sec. 11.1-11.2.3, PRML Sec. 9.2 slides (print version)
Aug 31 Gaussian Mixture Models (GMM) and Parameter Estimation for GMM PRML 9.2 - 9.3.2, 9.4 slides (print version), (notes)
Sept 5 The Expectation Maximization Algorithm PRML 9.3-9.4, MLAPP Chapter 11, Optional paper reading slides (print version)
Sept 7 Latent Variable Models for Dimensionality Reduction PRML Section 12.2, MLAPP Chapter 12 slides (print version)
Approximate Inference
Sept 12 Locally (Conditionally) Conjugate Models slides (print version)
Sept 14 Approximate Inference: Sampling Methods (1) PRML Chap. 11 (up to 11.1.4), MLAPP Chap. 23 (up to 23.4.2) slides (print version)
Oct 3 Approximate Inference: Sampling Methods (2) PRML Sec 11.2, 11.3, MLAPP Sec 24.1-24.3, Recommended: A detailed intro to MCMC, Gibbs Sampling slides (print version)
Oct 5 Approximate Inference: Sampling Methods (3) PRML Sec 11.2, 11.3, MLAPP Sec 24.1-24.3. Recommended: Gibbs Sampling, MCMC for Bayesian Matrix Factorization slides (print version)
Oct 10 Approximate Inference: Variational Bayes Inference (1) PRML Sec 10-10.1, 10.3-10.4, MLAPP 21.1-21.3, 21.5. Recommended: Variational Inference Review, Life after EM slides (print version)
Oct 12 Approximate Inference: Variational Bayes Inference (2) Optional but recommended: Section 5.1-5.2 of this monograph, BBVI paper, SVI paper slides (print version)
Assorted Topics
Oct 17 Learning Nonlinear Functions via Gaussian Processes (1) MLAPP Sections 15.1-15.2.5, (Optional: 15.3-15.5) slides (print version)
Oct 24 Learning Nonlinear Functions via Gaussian Processes (2) MLAPP Sections 15.1-15.2.5, 15.5 (Optional: 15.3-15.4), Optional: GPLVM Paper, Deep GP Paper slides (print version)
Oct 26 Probabilistic Topic Models Topic Models Intro 1, Topic Models Intro 2, Optional but recommended: Original LDA Paper slides (print version)
Oct 31 Deep Probabilistic Models (1) Recommended: Chapter 1 and 6 of this tutorial, Optional: Weight Uncertaintly in Neural Networks slides (print version)
Nov 2 Deep Probabilistic Models (2) Recommended: Chapter 6 of this tutorial, VAE paper, Optional: Deep Exp. Family paper, GAN paper slides (print version)
Nov 4 Nonparametric Bayesian Models for Unsupervised Learning Nonparametric Bayesian Models Survey, The IBP paper slides (print version)
Nov 7 Latent Variable Models for Sequential/Time-Series Data Recommended: (Sections relevant to LDS) from PRML Chapter 13, MLAPP Chapter 18 slides (print version)
Nov 9 Probabilistic Graphical Models, Inference via Message-Passing PRML Chapter 8.2-8.4.4 slides (print version)
Nov 14 Overview of Other Topics, Conclusion and Perspectives No readings.. slides

Reference materials

We will primarily use lecture notes/slides from this class. In addition, we will refer to monographs and research papers for some of the topics. Some recommended, although not required, books are: