Machine Learning Techniques
CS771: Machine Learning Techniques
Pre-requisites: MSO201A/equivalent, CS210/ESO211/ESO207A. Desirable: Familiarity with programming in MATLAB/Octave, Python, or R
About the course: Machine Learning is the discipline of designing algorithms that allow machines (e.g., a computer) to learn patterns and concepts from data without being explicitly programmed. This course will be an introduction to the design (and some analysis) of Machine Learning algorithms, with a modern outlook focusing on recent advances, and examples of real-world applications of Machine Learning algorithms. A tentative list of topics includes:
- Supervised Learning (Regression/Classification)
- Basic methods: Distance-based methods, Nearest-Neighbors, Decision Trees, Naı̈ve Bayes
- Linear models: Linear Regression, Logistic Regression, Generalized Linear Models
- Support Vector Machines, Nonlinearity and Kernel Methods
- Beyond Binary Classification: Multi-class/Structured Outputs, Ranking
- Unsupervised Learning
- Clustering: K-means/Kernel K-means
- Dimensionality Reduction: PCA and kernel PCA
- Matrix Factorization and Matrix Completion
- Generative Models (mixture models and latent factor models)
- Assorted Topics
- Evaluating Machine Learning algorithms and Model Selection
- Introduction to Statistical Learning Theory
- Ensemble Methods (Boosting, Bagging, Random Forests)
- Sparse Modeling and Estimation
- Modeling Sequence/Time-Series Data
- Deep Learning and Feature Representation Learning
- Scalable Machine Learning (Online and Distributed Learning)
- A selection from some other advanced topics, e.g., Semi-supervised Learning, Active Learning, Reinforcement Learning, Inference in Graphical Models, Introduction to Bayesian Learning and Inference
Reference materials: There will not be any dedicated textbook for this course. In lieu of that, we will have lecture slides/notes, and monographs, tutorials, and research papers for the topics that will be covered in this course. Some recommended (although not required) books are:
- Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012
- Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, Springer 2009 (freely available online)
- Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2007.
- Hal Daumé III, A Course in Machine Learning, 2015 (freely available online).