Topics in Probabilistic Modeling and Inference
2018-19 (even semester)

Instructor: Piyush Rai: (office: RM-502, email: piyush AT cse DOT iitk DOT ac DOT in)
Office Hours: Friday 6:00pm-7:00pm (or by appointment)
Q/A Forum: Piazza
Class Venue: KD-101
Class Timings: M/W 5:10-6:30pm
TAs: Shivam Bansal, Dhanajit Brahma, Abhishek Kumar, Vinay Verma
(Information on TA office hours and office locations will be posted on Piazza)

Background and Course Description

Probabilistic models for data are ubiquitous in many areas of science and engineering, and specific domains such as visual and language understanding, finance, healthcare, biology, climate informatics, etc. This course will be an advanced introduction to probabilistic models of data (often through case studies from these domains) and a deep-dive into advanced inference and optimization methods used to learn such probabilistic models. This is an advanced course and ideally suited for student who are doing research in this area or are interested in doing research in this area.


Instructor's consent. The course expects students to have a strong prior background in machine learning (ideally through formal coursework), and ideally also some prior exposure/appreciation to basic principles of probabilistic modeling. The students are expected to have strong foundations in probability and statistics, linear algebra, and optimization, and must also be proficient in programming in MATLAB, Python, or R.

Schedule (Tentative)

Date Topics Readings/References/Comments Slides/Notes
Jan 7 Course Logistics, Intro to Probabilistic Modeling and Inference [Z15], [B14] (for now, up to sec 3), a brief prob-stats refresher, a basic tutorial on Bayesian inference slides (print version)
Jan 9 Basics of Probabilistic/Bayesian Modeling and Parameter Estimation Wikipedia entries (to be read in the same order) on Bayesian Inference, Prior, Likelihood, Posterior, Posterior Predictive, Credible Intervals (for now, these articles are meant for cursory reading; may safely skip the parts that seem too advanced to you), Additional Reading: MLAPP Section 3.1-3.3 slides (print version)
Jan 12 Bayesian Inference for Some Basic Models MLAPP 3.3-3.5, Bayesian Inference for Gaussians, Wikipedia entry on Dirichlet distribution slides (print version)
Jan 14 Bayesian Inference for Gaussians, Working With Gaussians MLAPP 4.3-4.6 (it is far more detailed than you probably need at the moment; you may skip very detailed proofs, can focus more on the examples and the standard results on Gaussian properties, inference, etc), PRML 2.3, Bayesian Inference for Gaussians slides (print version)
Jan 16 Exponential Family Distributions and Conditional Models MLAPP 9.1-9.2, some notes on exp-family (if further interested, may skim through the Wikipedia article on exp-fam), PRML 3.3, MLAPP 7.1-7.3, 7.6 (7.6.1-7.6.2) slides (print version)

Suggested/Further Readings

- [Z15] Probabilistic machine learning and artificial intelligence: Zoubin Ghahramani
- [B14] Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models: David Blei

Useful Links and Softwares

  • Reference texts (locally accessible)
  • (more links coming soon..)

    Course Policies

    Anti-cheating policy