| Date |
Topics |
Readings/References/Comments |
Slides/Notes |
| Jan 7 |
Course Logistics, Intro to Probabilistic Modeling and Inference |
[Z15], [B14] (for now, up to sec 3), a brief prob-stats refresher, a basic tutorial on Bayesian inference |
slides (print version) |
| Jan 9 |
Basics of Probabilistic/Bayesian Modeling and Parameter Estimation |
Wikipedia entries (to be read in the same order) on Bayesian Inference, Prior, Likelihood, Posterior, Posterior Predictive, Credible Intervals (for now, these articles are meant for cursory reading; may safely skip the parts that seem too advanced to you), Additional Reading: MLAPP Section 3.1-3.3, Conjugate Priors |
slides (print version) |
| Jan 12 |
Bayesian Inference for Some Basic Models |
Lecture 2 suggested readings + MLAPP 3.3-3.5, Bayesian Inference for Gaussians, Wikipedia entry on Dirichlet distribution |
slides (print version) |
| Jan 14 |
Bayesian Inference for Gaussians, Working With Gaussians |
MLAPP 4.3-4.6 (it is far more detailed than you probably need at the moment; you may skip very detailed proofs, can focus more on the examples and the standard results on Gaussian properties, inference, etc), PRML 2.3, Bayesian Inference for Gaussians |
slides (print version) |
| Jan 16 |
Exponential Family Distributions and Conditional Models |
MLAPP 9.1-9.2, some notes on exp-family (if further interested, may skim through the Wikipedia article on exp-fam), PRML 3.3, MLAPP 7.1-7.3, 7.6 (7.6.1-7.6.2) |
slides (print version) |
| Jan 21 |
Bayesian Linear Regression (Hyperparameter Estimation, Sparse Priors), Bayesian Logistic Regression |
Recommended Readings: Bayesian Inference tutorial (with Bayesian linear regression as case study), Relevance Vector Machine (note: both these papers are nice readings in the sense that they put together a lot of ideas that we have seen thus far in a concrete model - Bayesian linear regression with possibly sparse weights. These papers may still appear a bit "advanced" at the moment since some of the things you would find in these papers have not been introduced to you yet and you can skim over those parts. But you are nevertheless encouraged to read these papers at a high level to get a holistic view of probabilistic/Bayesian modeling and inference, at least in simple models) |
slides (print version) |
| Jan 23 |
Bayesian Logistic Regression, Laplace Approximation, Bayesian Generative Classification |
MLAPP Sec 8.4 (Bayesian Logistic Regression), Sec 3.5.1.2, 3.5.2, 3.5.5 (for some examples of generative classification, including the Bayesian way) |
slides (print version) |
| Jan 28 |
Gaussian Processes for Learning Nonlinear Functions |
PRML Sec. 6.4, MLAPP Sections 15.1-15.2.5, (Optional: 15.3-15.4), Illustration of various kernels for GP, Some GP software packages: GPFlow (Tensorflow based), GPyTorch (PyTorch based), GPML (MATLAB based) |
slides (print version) |
Jan 30 |
Gaussian Processes (Contd.) |
PRML Sec. 6.4, MLAPP Sections 15.1-15.2.5, (Optional: 15.3-15.4), Illustration of various kernels for GP, Some GP software packages: GPFlow (Tensorflow based), GPyTorch (PyTorch based), GPML (MATLAB based) |
slides (print version) |
| Feb 4 |
Inference in Multiparameter Models, Conditional Posterior, Local Conjugacy |
Highly recommended: Paper on Bayesian Matrix Factorization, and Gibbs Sampling for the Uninitiated (note: we will look at Gibb sampling again in more detail and formally when talking about MCMC but if you want to get a good and practical overview then this tuutorial is very nice and doesn't require you to understand MCMC in much detail beforehand) |
slides (print version) |
| Feb 9 |
Latent Variable Models (LVMs) and Inference in LVMs |
MLAPP 11.4, Optional readings: Original EM paper (technically very dense but lots of interesting insights), Another classic paper on EM (more accessible), Online EM (practically oriented), Online EM (theoretically oriented) |
slides (print version) |
| Feb 11 |
Expectation-Maximization (Contd) and Introduction to Variational Inference |
Online EM paper (recommended), Reading on VI: PRML 10.1 |
slides (print version) |
| Feb 13 |
Variational Inference (Contd) |
Readings: PRML 10.1,10.2,10.3.10.4, VI: A Review for Statisticians (up to Sec 4.2 for now) |
slides (print version) |
| Feb 25 |
Variational Inference: Recent Advances |
Readings: VI: A Review for Statisticians (Sec 4.3 on SVI), SVI paper (if you are interested in a more in-depth treatment of SVI), Advances in Variational Inference (a bit long but I would suggest skimming it over to get a sense of the various recent advances in VI) |
slides (print version) |
| Feb 27 |
VI (wrap-up), Inference via Sampling |
Readings: PRML Sec 11-11.1, Also recommended: Advances in Variational Inference, Monte Carlo for Absolute Beginners (up to Sec 2) - Discusses some classic sampling methods in detail and simple examples |
slides (print version) |
| March 2 |
Inference via Sampling (Contd) |
Readings: PRML 11.1-11.3, MLAPP 24.1-24.3, Recommended: Monte Carlo for Absolute Beginners, Another old but detailed intro to MCMC |
slides (print version) |
| March 6 |
Inference via Sampling (Contd), Gradient-based and Online MCMC |
Readings: PRML 11.1-11.3, MLAPP 24.1-24.4, Recommended: SGLD paper, and the other two recommended papers for previous lecture |
slides (print version) |
| March 11 |
Gradient-based and Online Sampling Methods, Recent Advances in Sampling Methods |
Recommended: SGLD paper and other papers referenced in the slides (not required to get into every technical detail but try skimming through some of these papers to get a high level idea), Patterns of Scalable Bayes (See sec 4.2 for parallel MCMC, though other parts are also useful for a general introduction to approximate inference methods, including scalable methods), Some HMC references: HMC (a brief tutorial), No U-Turn Sampler (explains HMC and also describes how to eliminate the need of L leapfrog steps in HMC), HMC (an in-depth introduction) |
slides (print version) |
| March 13 |
Probabilistic Topic Models |
Recommended: A brief intro to topic Models (basic LDA, experimental examples, and some extensions), Poisson Matrix Factorization and LDA (Sec 4 and 5 shows how LDA and other closely related models are related to Poisson matrix factorization) |
slides (print version) |
| March 25 |
Probabilistic Models for Graphs, and Intro to Nonparametric Bayesian Modeling |
Basic readings: MLAPP Sec 27.5 (Prob. models of graphs), MLAPP Sec 25.2 (for NPBayes Clustering), Other recommended readings: For prob. models of graphs - papers listed in the slides; For NPBayes modeling general intro - this tutorial survey paper, For NPBayes Clustering, this paper on Dirichlet Process |
slides (print version) |
| March 27 |
Nonparametric Bayesian Modeling (Contd) |
MLAPP Sec 25.2 (for NPBayes Clustering), Other recommended readings: For NPBayes modeling general intro - this tutorial survey paper, For NPBayes Clustering, this paper on Dirichlet Process |
slides (print version) |
| April 1 |
Nonparametric Bayesian Modeling (Wrap-up) |
MLAPP Sec 25.2 (for NPBayes Clustering), Other recommended readings: For NPBayes modeling general intro - this tutorial survey paper, For NPBayes Clustering, this paper on Dirichlet Process |
slides (print version) |
| April 3 |
Probabilistic Modeling meets Deep Learning |
Recommended Readings: Papers listed in the slides |
slides (print version) |
| April 8 |
Probabilistic Deep Learning (Wrap-up), Latent Variable Models for Sequential Data |
Recommended Readings: PRML Chapter 13 (sections relevant to LDS; sections on HMM optional), MLAPP Chapter 18 |
slides (print version) |
| April 10 |
Probabilistic Graphical Models, Inference via Message-Passing |
PRML Sec 8.2-8.4.4 |
slides (print version) |
| April 15 |
Sequential Decision-Making under Uncertainty (Active Learning, Bayesian Optimization, Bandits) |
Recommended Readings: Papers listed in the slides |
slides (print version) |
