## Topics in Probabilistic Modeling and Inference |

Instructor: Piyush Rai: (office: RM-502, email: piyush AT cse DOT iitk DOT ac DOT in)

Office Hours: Wednesday 11:00am-12:00pm (or by appointment)

Class Venue: KD-101

Class Timings: T/Th 5:10-6:30pm

TAs: Shivam Bansal, Smrithi Prabhu, Vinay Verma

TA Office Hours: TBD

Date |
Topics |
Readings/References |
Slides/Notes |

Jan 4 | Logistics and Introduction to the course | Nature article on probabilistic modeling, Probability Refresher slides | slides, (print version) |

Jan 9 | Basics of Probabilistic Modeling and Inference, Single-Parameter Models | Parameter Estimation (only up to section 3), BDA Section 1.1-1.3, 1.8, BDA 2.1-2.6 | slides, (print version) |

Jan 11 | Single-Parameter Models (Contd.), Intro to Bayesian Linear Regression | BDA 2.1-2.6, MLAPP 7.1-7.3, 7.6 (7.6.1-7.6.2) | slides, (print version) |

Jan 11 | Bayesian Linear Regression (Contd) | PRML 3.3, MLAPP 7.1-7.3, 7.6 (7.6.1-7.6.2) | slides, (print version) |

A tentative set of topics to be covered includes:

- Fundamentals of probabilistic modeling
- Basics of probability distributions and their properties
- Basics of probabilistic inference: MLE/MAP/Bayesian inference
- Probabilistic graphical models (directed and undirected models)
- Hierarchical modeling, multi-parameter models
- Bayesian vs frequentist statistics
- Probabilistic approaches for linear modeling, Sparse Bayesian Learning
- Latent variable models
- Mixture models and latent factor models
- Latent variable models for dynamic/sequential data
- Latent variable models for networks and relational data
- Latent variable models with covariates
- Approximate Inference
- Inference in probabilistic graphical models
- MCMC methods
- Variational methods
- Scalable inference with stochastic optimization
- Other methods: Likelihood-free methods, spectral methods, etc.
- Nonparametric Bayesian methods
- Gaussian Process for function approximation
- Dirichlet process and beta processes
- Other stochastic processes (gamma/point processes, etc., and applications)
- Bayesian Optimization
- Probabilistic programming
- Other topics based on students' interests

- (PRML) Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2007.
- (MLAPP) Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
- (BDA) Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, Donald B. Rubin. Bayesian Data Analysis, Chapman \& Hall/CRC, 2013
- (ITILA) David Mackay. Information Theory, Inference, and Learning Algorithms. Cambridge Univ. Press, 2003.
- Papers from conference/journals in machine learning and Bayesian statistics (e.g., ICML, NIPS, AISTATS, Journal of Machine Learning Research, Machine Learning Journal, Bayesian Analysis, Biometrika, Annals of Statistics, etc.)