CS 771: Machine Learning Techniques

Pre-requisites:

MSO201A/equivalent, CS210/ESO211/ESO207A. Desirable: Familiarity with programming in MATLAB/Octave, Python, or R

About the course:

Machine Learning is the discipline of designing algorithms that allow machines (e.g., a computer) to learn patterns and concepts from data without being explicitly programmed. This course will be an introduction to the design (and some analysis) of Machine Learning algorithms, with a modern outlook focusing on recent advances, and examples of real-world applications of Machine Learning algorithms. A tentative list of topics includes:

1. Supervised Learning (Regression/Classification)
   1. Basic methods: Distance-based methods, Nearest-Neighbors, Decision Trees, Naı̈ve Bayes
   2. Linear models: Linear Regression, Logistic Regression, Generalized Linear Models
   3. Support Vector Machines, Nonlinearity and Kernel Methods
   4. Beyond Binary Classification: Multi-class/Structured Outputs, Ranking
2. Unsupervised Learning
   1. Clustering: K-means/Kernel K-means
   2. Dimensionality Reduction: PCA and kernel PCA
   3. Matrix Factorization and Matrix Completion
   4. Generative Models (mixture models and latent factor models)
3. Assorted Topics
   1. Evaluating Machine Learning algorithms and Model Selection
   2.  Introduction to Statistical Learning Theory
   3.  Ensemble Methods (Boosting, Bagging, Random Forests)
   4.  Sparse Modeling and Estimation
   5.  Modeling Sequence/Time-Series Data
   6.  Deep Learning and Feature Representation Learning
   7. Scalable Machine Learning (Online and Distributed Learning)
   8. A selection from some other advanced topics, e.g., Semi-supervised Learning, Active Learning, Reinforcement Learning, Inference in Graphical Models, Introduction to Bayesian Learning and Inference

Reference materials

There will not be any dedicated textbook for this course. In lieu of that, we will have lecture slides/notes, and monographs, tutorials, and research papers for the topics that will be covered in this course. Some recommended (although not required) books are:

1. Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012
2. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, Springer 2009 (freely available online)
3. Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2007.
4. Hal Daumé III, A Course in Machine Learning, 2015 (freely available online).