Probabilistic Machine Learning
CS772A
2017-18 (odd semester)


Instructor: Piyush Rai: (office: RM-502, email: piyush AT cse DOT iitk DOT ac DOT in)
Office Hours: Wed 11:00-12:00 (or by appointment)
Q/A Forum: Piazza
Class Venue: RM-101
Class Timings: Tue/Th 6:00-7:15pm
TAs: Gundeep Arora, Smrithi Prabhu, Vinay Verma, Prem Raj
TA Office Hours: Gundeep (Fri, 3-4pm, RM 515), Smrithi (Mon 3-4pm, RM 515), Vinay (Tue 4-5pm, RM 504)

Pre-requisites

CS771 (Intro to Machine Learning) or equivalent. This requirement can be waived if you have a significant prior exposure to machine learning through course-work or substantial project-work. Note that this course will make extensive use of probability, statistics, and optimization. A solid background on these topics, as well as introductory machine learning, will be essential. Students are also supposed to be familiar with programming in MATLAB/Python.

Grading

(Tentative break-up) There will be 5 pen-and-paper assignments (total 30%), a mid-term (20%), a final-exam (25%), and a course-project (25%). A bonus of up to 5% will be reserved for class/Piazza participation.

Schedule (Tentative)

(*) Re-scheduled class
Date Topics Readings/References Slides/Notes
Aug 1 Course Logistics and Introduction Nature article, Probability Refresher slides slides (print version)
Foundations and Probabilistic Supervised Learning
Aug 3 Basics of Parameter Estimation in Probabilistic Models Parameter Estimation (only up to Section 3.1 for now) slides (print version)
Aug 5 (*) Probabilistic Models for Regression MLAPP Section 7.1-7.3, 7.5.1, 7.6 (up to 7.6.2) slides (print version)
Aug 17 Probabilistic Models for Classification (I): Generative Classification Optional Readings: PRML Section 4.2, MLAPP Section 4.1-4.2.5 slides (print version)
Aug 19 (*) Probabilistic Models for Classification (II): Discriminative Classification Optional Readings: PRML Section 4.3, MLAPP Sections 8.1-8.4, 8.6 slides (print version)
Aug 22 Exponential Family and Generalized Linear Models MLAPP Sections 9.1-9.3, Exponential Family and GLMs slides (print version)
Aug 24 Hyperparameter Estimation in Probabilistic Models PRML Section 3.5 slides (print version)
Aug 26 (*) Working with Gaussians, Linear Gaussian Models MLAPP Sec. 4.1, 4.3-4.4, PRML Sec. 2.3 slides (print version)
Simple Latent Variable Models
Aug 29 Introduction to Latent Variable Models, LVMs for Clustering MLAPP Sec. 11.1-11.2.3, PRML Sec. 9.2 slides (print version)
Aug 31 Gaussian Mixture Models (GMM) and Parameter Estimation for GMM PRML 9.2 - 9.3.2, 9.4 slides (print version), (notes)
Sept 5 The Expectation Maximization Algorithm PRML 9.3-9.4, MLAPP Chapter 11, Optional paper reading slides (print version)
Sept 7 Latent Variable Models for Dimensionality Reduction PRML Section 12.2, MLAPP Chapter 12 slides (print version)
Approximate Inference
Sept 12 Locally (Conditionally) Conjugate Models slides (print version)
Sept 14 Approximate Inference: Sampling Methods (1) PRML Chap. 11 (up to 11.1.4), MLAPP Chap. 23 (up to 23.4.2) slides (print version)

Topics

A tentative set of topics to be covered includes (note: topics in red denote the key differences from my previous offering of this course):

Reference materials

We will primarily use lecture notes/slides from this class. In addition, we will refer to monographs and research papers for some of the topics. Some recommended, although not required, books are: