Introduction to Machine Learning
CS771A
Autumn 2023

Instructor: Piyush Rai: (office: RM-502, email: piyush AT cse DOT iitk DOT ac DOT in)
TAs: Aditya Dhaulakhandi (adityad@cse), Subhajit Panday (subhajitpanday@cse), Malay Pandey (malay@cse), Abhishek Jaiswal (abhijais@cse), Putrevu Venkata Sai Charan (pvcharan@cse), Pramit Bhattacharyya (pramitb@cse), Gargi Sarkar (gsarkar@cse), Virendra Nishad (viren@cse), Priyanka Maity (priyankamaity@cse), Ayush Pande (ayushp@cse), Debkanta Chakraborty (debkanta@cse), Saqib Sarwar (saqib@cse)
TA Office Hours: TBA
Q/A Forum: Piazza
Class Location: L-20 (lecture hall complex)
Timings: Mon/Thur 6:00-7:30pm

Background and Course Description

Machine Learning is the discipline of designing algorithms that allow machines (e.g., a computer) to learn patterns and concepts from data without being explicitly programmed. This course will be an introduction to the design (and some analysis) of Machine Learning algorithms, with a modern outlook, focusing on the recent advances, and examples of real-world applications of Machine Learning algorithms. This is supposed to be the first ("intro") course in Machine Learning. No prior exposure to Machine Learning will be assumed. At the same time, please be aware that this is NOT a course about toolkits/software/APIs used in applications of Machine Learning, but rather on the principles and foundations of Machine Learning algorithms, delving deeper to understand what goes on "under the hood", and how Machine Learning problems are formulated and solved.

Grading

Tentative grading scheme: There will be 4 quizzes (worth 30% of your grade), 2 homework assignments (worth 20%), a mid-sem exam (worth 20%), and a final exam (worth 30%).

Reference materials

There will not be any dedicated textbook for this course. Recommended readings for specific topics will be suggested. Some reference books (free PDF available for most) for machine learning are listed below:

Kevin P. Murphy. Probabilistic Machine Learning: An Introduction (PML), MIT Press, March 2022.
Christopher Bishop, Pattern Recognition and Machine Learning (PRML), Springer, 2007.
Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning (ESL), Springer, 2009 (freely available online)
David G. Stork, Peter E. Hart, and Richard O. Duda. Pattern Classification (PC), Wiley-Blackwell, 2000
Aston Zhang, Zack C. Lipton, Mu Li, Alex J. Smola. Dive into Deep Learning (D2L), Cambridge University Press, 2023.
Simon J.D. Prince. Understanding Deep Learning (UDL), MIT Press, 2023.
Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong. Maths for Machine Learning (MML), Cambridge University Press, 2023.

Schedule

Note: PPTX slides best viewed in Microsoft Powerpoint

Lec. No.	Topics	Readings/References	Code/Demo	Slides/Notes
1	Course Logistics and Introduction to Machine Learning	ML article in Science, maths refresher slides	Python basics, NumPy basics	PPTX slides, PDF slides
2	Data and Features, Supervised Learning by Computing Distances (Learning with Prototypes)	Learning with Prototypes	LwP demo	PPTX slides, PDF slides
3	Learning with Prototypes (wrap-up), Nearest Neighbors	(Optional but recommended) sklearn page on nearest neighbors	KNN demo	PPTX slides, PDF slides
4	Learning with Decision Trees	Intro to DT, A nice visual illustration of DTs, sklearn page on Decision Trees		PPTX slides, PDF slides
5	Linear Models	Least Squares and Ridge Regression	Ridge regression: demo 1, demo 2	PPTX slides, PDF slides
6	Linear Models (contd), Linear Classification (logistic and softmax)	Section 10.1-10.3 from PML (logistic and softmax/multinomial classification)	Logistic Regression, Softmax Classification	PPTX slides, PDF slides
7	Optimization Techniques for ML	Chapter 8 from PML (only the topics discussed in class). Also recommended: This blog post		PPTX slides, PDF slides
8	Optimization Techniques for ML (contd)	Same readings as lecture 7	GD vs ALT-OPT	PPTX slides, PDF slides
9	Optimization Techniques for ML (wrap-up)	Same readings as lecture 7		PPTX slides, PDF slides
10	Large-Margin Classification: Support Vector Machines	PML 17.3	Linear SVM (in dual): Hard-Margin, Soft-Margin,	PPTX slides, PDF slides
11	Support Vector Machines (contd)	PML 17.3	Linear SVM (in primal): Hinge-Loss Optimization	PPTX slides, PDF slides
12	Kernel Methods	PML 17.1 and 17.3	Kernel LwP, Kernel SVM	PPTX slides, PDF slides
13	Probabilistic Modeling: The Basics	Probability basics for ML: PPTX slides, PDF slides		PPTX slides, PDF slides
14	Probabilistic Models for Supervised Learning: Discriminative Methods			PPTX slides, PDF slides
15	Probabilistic Models for Supervised Learning: Generative Methods	PML-1: Chapter 9		PPTX slides, PDF slides
16	Unsupervised Learning: Clustering	PRML (Bishop) Section 9.1, PML-1: Section 21.1-21.3	K-means, K-means++	PPTX slides, PDF slides
17, 18	Unsupervised Learning: Dimensionality Reduction	PML-1: Section 20.1, 20.4, 20.5		PPTX slides, PDF slides
19, 20	Probabilistic Models for Unsupervised Learning: Latent Variable Models	PRML 9.2 - 9.3.2, 9.4, Sec 12.2 (up to 12.2.2)		PPTX slides, PDF slides
21	Intro to Deep Neural Networks: Multi-layer Perceptrons	PML-1 Chapter 13		PPTX slides, PDF slides
22	Intro to Deep Neural Nets (Contd)	PML-1 Chapter 13		PPTX slides, PDF slides
23	Beyond MLPs: Convolutional Neural Networks	PML-1 Chapter 14		PPTX slides, PDF slides
24	Beyond MLPs: Deep Neural Networks for Sequential Data	PML-1 Section 15.1-15.3		PPTX slides, PDF slides
25	Attention Mechanism and Transformers	PML-1 Section 15.4-15.7		PPTX slides, PDF slides
26	Deep Neural Networks: Some Assorted Topics	Papers references on the slides		PPTX slides, PDF slides
27	The Last Few Bits			PPTX slides, PDF slides

Course Policies

Anti-cheating policy

Introduction to Machine Learning CS771A Autumn 2023