Probabilistic Machine Learning
CS772A
2024-25 (even semester)
Class Timings: Mon/Thur 18:15 - 19:30
Class Venue: RM-101
Instructor: Piyush Rai (Office RM-502, email: piyush AT cse DOT iitk DOT ac DOT in)
Office Hours: By appointment (RM-502)
Q/A Forum: Piazza

Background and Course Description

Probabilistic models for data are ubiquitous in many areas of science and engineering, and specific domains such as visual and language understanding, finance, healthcare, biology, climate informatics, etc. This course will be an advanced introduction to probabilistic models of data (often through case studies from these domains) and a deep-dive into advanced inference and optimization methods used to learn such probabilistic models. This is an advanced course and ideally suited for student who are doing research in this area or are interested in doing research in this area.

Pre-requisites

Instructor's consent. The course expects students to have a strong prior background in machine learning (ideally through formal coursework, such as CS771), and ideally also some prior exposure/appreciation to basic principles of probabilistic modeling. The students are expected to have strong foundations in probability and statistics, linear algebra, and optimization, and must also be proficient in programming in Python.

Grading

(Tentative) There will be 3 quizzes (30%), a mid-sem exam (20%), an end-sem exam (30%), and a research project (20%).

Reference materials

There will not be any dedicated textbook for this course. We will use lecture slides/notes, monographs, tutorials, and papers for the topics that will be covered in this course. Some recommended, although not required, reference books are listed below:

Schedule

Lec. No. Topics Readings/References/Comments Slides/Notes
1 Course Logistics, Intro to Probabilistic Machine Learning A review article on PML and AI, Probability and statistics refresher slides PPTX slides, PDF slides
2 Probabilistic ML: Some Basic Ideas PML-1 (Section 4.6), Optional/recommended: Chapter 2 and Appendix B of PRML PPTX slides, PDF slides
3 Estimating Parameters and Predictive Distributions: Some Simple Cases PML-1 (Section 4.6), Optional/recommended: Chapter 2 and Appendix B of PRML PPTX slides, PDF slides
4 Gaussian Observation Model: Some Examples PML-1 (Section 4.6, Section 11.7). Optional/recommended: Conjugate Bayesian analysis of the Gaussian distribution PPTX slides, PDF slides
5 Probabilistic Linear Regression Bayesian inference tutorial (Section 1-3) PPTX slides, PDF slides
6 Logisti/Softmax Classification, Laplace Approximation PML-2 (Section 15.3) PPTX slides, PDF slides
7 Model Selection and Model Averaging, Exponential Family PML-2: Section 2.3, 2.4, 3.4.5 PPTX slides, PDF slides
8 Exponential Family (contd), Generative Supervised Learning CS771 slides on the generative sup. learning (PPTX, PDF), PML-1 (Chapter 9) PPTX slides, PDF slides
9 Gaussian Process (GP) PML-2 Chapter 18 PPTX slides, PDF slides
10 GP (wrap-up), Latent Variable Models and EM PRML Chapter 9 PPTX slides, PDF slides
11 LVMs and EM (contd) PRML Chapter 9 PPTX slides, PDF slides
12 EM (wrap-up) PRML Chapter 9 (+ PRML Section 3.5 on MLE-II for Bayesian Linear Regression) PPTX slides, PDF slides
13 Variational Inference PML-2 Chapter 10.1-10.4 PPTX slides, PDF slides
14 Variational Inference (contd) PML-2 Chapter 10.1-10.4 PPTX slides, PDF slides
15 Variational Inference (wrap-up), Sampling from distributions PML-2 Chapter 10.1-10.4, PML-2: Chapter 11.1-11.5 PPTX slides, PDF slides
16 Sampling Methods, MCMC PPTX slides, PDF slides
17 MCMC (contd) PPTX slides, PDF slides
18 MCMC (contd) PPTX slides, PDF slides
19 MCMC (wrap-up), Deep Generative Models PPTX slides, PDF slides
20 Deep Generative Models (VAE and GAN) PPTX slides, PDF slides
21 Denoising Diffusion Models PPTX slides, PDF slides
22 Denoising Diffusion Models (contd) PPTX slides, PDF slides
23 Large Language Models (Autoregressive and Diffusion-based) PPTX slides, PDF slides
24 Active Learning and Bayesian Optimization PPTX slides, PDF slides
25 Assorted Topics (1): Calibration, Frequentist Statistics PPTX slides, PDF slides
26 Assorted Topics (2): Conformal Prediction, Simulation-based Inference PPTX slides, PDF slides

Course Policies

Anti-cheating policy