CS771: Machine Learning Techniques

Pre-requisites: MSO201A/equivalent, CS210/ESO211/ESO207A. Desirable: Familiarity with programming in MATLAB/Octave, Python, or R

About the course: Machine Learning is the discipline of designing algorithms that allow machines (e.g., a computer) to learn patterns and concepts from data without being explicitly programmed. This course will be an introduction to the design (and some analysis) of Machine Learning algorithms, with a modern outlook focusing on recent advances, and examples of real-world applications of Machine Learning algorithms. A tentative list of topics includes:

• Supervised Learning (Regression/Classification)
  • Basic methods: Distance-based methods, Nearest-Neighbors, Decision Trees, Naı̈ve Bayes
  • Linear models: Linear Regression, Logistic Regression, Generalized Linear Models
  • Support Vector Machines, Nonlinearity and Kernel Methods
  • Beyond Binary Classification: Multi-class/Structured Outputs, Ranking
• Unsupervised Learning
  • Clustering: K-means/Kernel K-means
  • Dimensionality Reduction: PCA and kernel PCA
  • Matrix Factorization and Matrix Completion
  • Generative Models (mixture models and latent factor models)
• Assorted Topics
  • Evaluating Machine Learning algorithms and Model Selection
  •  Introduction to Statistical Learning Theory
  •  Ensemble Methods (Boosting, Bagging, Random Forests)
  •  Sparse Modeling and Estimation
  •  Modeling Sequence/Time-Series Data
  •  Deep Learning and Feature Representation Learning
  • Scalable Machine Learning (Online and Distributed Learning)
  • A selection from some other advanced topics, e.g., Semi-supervised Learning, Active Learning, Reinforcement Learning, Inference in Graphical Models, Introduction to Bayesian Learning and Inference

Reference materials: There will not be any dedicated textbook for this course. In lieu of that, we will have lecture slides/notes, and monographs, tutorials, and research papers for the topics that will be covered in this course. Some recommended (although not required) books are:

• Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012
• Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, Springer 2009 (freely available online)
• Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2007.
• Hal Daumé III, A Course in Machine Learning, 2015 (freely available online).