Course Calendar by Week

Deliverables with Deadlines

Tutorial Schedule

Detailed Calendar by Session

• Event
  Date
  Description
  Description
• Session
  01/06/2026 13:00
  Tuesday
  First Lecture
• Lecture
  01/06/2026
  Tuesday
  Lecture 0: Course Overview and Logistics

  Lecture Notes:

  • Chapter 0
• Lecture
  01/06/2026
  Tuesday
  Lecture 1: Introduction and DL Components

  Lecture Notes:

  • Chapter 1 - Section 1
  • Chapter 1 - Section 2

  Further Reads:

  • Motivation: Chapter 1 - Section 1.1 of [BB]
  • Review on Linear Algebra: Chapter 2 of [GYC]
  • ML Components: Chapter 1 - Sections 1.2.1 to 1.2.4 of [BB]
• Lecture
  01/08/2026
  Thursday
  Lecture 2: Classification via Perceptron

  Lecture Notes:

  • Chapter 1 - Section 2

  Further Reads:

  • Binary Classification: Chapter 5 - Sections 5.1 and 5.2 of [BB]
  • McCulloch-Pitts Model: Paper A logical calculus of the ideas immanent in nervous activity published in the Bulletin of Mathematical Biophysics by Warren McCulloch and Walter Pitts in 1943, proposing a computational model for neuron. This paper is treated as the pioneer study leading to the idea of artificial neuron –>
• Lecture
  01/08/2026
  Thursday
  Lecture 3: Training via Empirical Risk Minimization

  Lecture Notes:

  • Chapter 1 - Section 2

  Further Reads:

  • Overview on Risk Minimization: Paper An overview of statistical learning theory published as an overview of his life-going developments in ML in the IEEE Transactions on Neural Networks by Vladimir N. Vapnik in 1999
• Lecture
  01/13/2026
  Tuesday
  Lecture 4: Multiple Layers of Perceptrons

  Lecture Notes:

  • Chapter 1 - Section 3

  Further Reads:

  • Perceptron Simulation Experiments: Paper Perceptron Simulation Experiments presented by Frank Rosenblatt in Proceedings of IRE in 1960
  • Perceptron: Chapter 1 - Section 1.2.1 of [Ag]
• Lecture
  01/13/2026
  Tuesday
  Lecture 5: Universal Approximation Theorem and Deep NNs

  Lecture Notes:

  • Chapter 1 - Section 3
  • Chapter 1 - Section 4

  Further Reads:

  • Universal Approximation: Paper Approximation by superpositions of a sigmoidal function published in Mathematics of Control, Signals and Systems by George V. Cybenko in 1989
  • DNNs: Chapter 6 - Sections 6.2 and 6.3 of [BB]
• Assignment
  01/15/2026
  Thursday
  Assignment #1 - Fundamentals of Computational Learning released!
  [Assignment #1 - Fundamentals of Computational Learning]
• Lecture
  01/20/2026
  Tuesday
  Lecture 6: Iterative Optimization by Gradient Descent

  Lecture Notes:

  • Chapter 1 - Section 5

  Further Reads:

  • Gradient-based Optimization: Chapter 4 - Sections 4.3 and 4.4 of [GYC]
  • Gradient Descent: Chapter 7 - Sections 7.1 and 7.2 of [BB]
• Lecture
  01/20/2026
  Tuesday
  Lecture 07: More on Gradient Descent

  Lecture Notes:

  • Chapter 1 - Section 5

  Further Reads:

  • Gradient Descent: Chapter 7 - Section 7.2 of [BB]
• Lecture
  01/22/2026
  Thursday
  Lecture 08: Forward Propagation in MLPs

  Lecture Notes:

  • Chapter 2 - Section 1

  Further Reads:

  • Deep FNNs: Chapter 6 - Sections 6.3 and 6.4 of [GYC]
• Lecture
  01/22/2026
  Thursday
  Lecture 09: Computing Gradient on Graph

  Lecture Notes:

  • Chapter 2 - Section 2

  Further Reads:

  • Backpropagation: Chapter 6 - Section 6.5 of [GYC]
• Lecture
  01/27/2026
  Tuesday
  Lecture 10: Backpropagation

  Lecture Notes:

  • Chapter 2 - Section 3

  Further Reads:

  • Backpropagation: Chapter 6 - Section 6.5 of [GYC]
  • Backpropagation: Chapter 8 of [BB]
• Lecture
  01/27/2026
  Tuesday
  Lecture 11: Backpropagation over MLP

  Lecture Notes:

  • Chapter 2 - Section 3

  Further Reads:

  • Backpropagation: Chapter 8 of [BB]
  • Backpropagation of Error Paper Learning representations by back-propagating errors published in Nature by D. Rumelhart, G. Hinton and R. Williams in 1986 advocating the idea of systematic gradient computation of a computation graph
• Assignment
  01/29/2026
  Thursday
  Assignment 2: MLPs released!
  [Assignment 2: MLPs]
• Lecture
  01/29/2026
  Thursday
  Lecture 12: Neural Classifier

  Lecture Notes:

  • Chapter 2 - Section 4

  Further Reads:

  • Binary Classification: Chapter 6 - Section 6.6 of [BB]
• Lecture
  01/29/2026
  Thursday
  Lecture 13: Multiclass Classification

  Lecture Notes:

  • Chapter 2 - Section 4

  Further Reads:

  • Multi-class Models: Chapter 2 - Section 2.3 of [Ag]
• Due
  01/29/2026 23:59
  Thursday
  Assignment #1 due
• Assignment
  01/30/2026
  Friday
  Project Proposal released!
  [Project Proposal]
• Lecture
  02/03/2026
  Tuesday
  Lecture 14: Stochastic Gradient Descent and Learning Curves

  Lecture Notes:

  • Chapter 2 - Section 5

  Further Reads:

  • SGD: Chapter 5 - Section 5.9 of [GYC]
• Lecture
  02/03/2026
  Tuesday
  Lecture 15: Linear and Sub-linear Convergence Speed

  Lecture Notes:

  • Chapter 3 - Section 1

  Further Reads:

  • Notes on Optimizers Lecture notes of the course Optimization for Machine Learning by Ashok Cutkosky in Boston University: A good resource for optimizers
• Lecture
  02/05/2026
  Thursday
  Lecture 16: Practical Optimizers

  Lecture Notes:

  • Chapter 3 - Section 1

  Further Reads:

  • Learning Rate Scheduling Paper Cyclical Learning Rates for Training Neural Networks published in Winter Conference on Applications of Computer Vision (WACV) by Leslie N. Smith in 2017 discussing learning rate scheduling
  • Rprop Paper A direct adaptive method for faster backpropagation learning: the RPROP algorithm published in IEEE International Conference on Neural Networks by M. Riedmiller and H. Braun in 1993 proposing Rprop algorithm
• Lecture
  02/05/2026
  Thursday
  Lecture 17: Overfitting and Regularization

  Lecture Notes:

  • Chapter 3 - Section 2

  Further Reads:

  • Regularization: Chapter 7 of [GYC]
• Due
  02/06/2026 23:59
  Friday
  Proposal Due
• Lecture
  02/10/2026
  Tuesday
  Lecture 18: Dropout and Data Augmentation

  Lecture Notes:

  • Chapter 3 - Section 2

  Further Reads:

  • Dropout 1 Paper Improving neural networks by preventing co-adaptation of feature detectors published in 2012 by G. Hinton et al. proposing Dropout
  • Dropout 2 Paper Dropout: A Simple Way to Prevent Neural Networks from Overfitting published in 2014 by N. Srivastava et al. providing some analysis and illustrations on Dropout
  • Data: Chapter 8 of the Book Patterns, predictions, and actions: A story about machine learning by Moritz Hardt and B. Recht published in 2021
  • Data Processing in Python Open Book Minimalist Data Wrangling with Python by Marek Gagolewski going through data processing in Python
• Lecture
  02/10/2026
  Tuesday
  Lecture 19: Normalization

  Lecture Notes:

  • Chapter 3 - Section 4

  Further Reads:

  • Data Processing in Python Open Book Minimalist Data Wrangling with Python by Marek Gagolewski going through data processing in Python
• Lecture
  02/12/2026
  Thursday
  Lecture 20: Batch Normalization

  Lecture Notes:

  • Chapter 3 - Section 4

  Further Reads:

  • Batch-Norm Paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift published in 2015 by S. Ioffe and C. Szegedy proposing Batch Normalization
  • Batch-Norm Meaning Paper How Does Batch Normalization Help Optimization? published in 2018 by S. Santurkar et al. discussing why Batch Normalization works: they claim that the main reason is that loss landscape is getting much smoother
• Lecture
  02/12/2026
  Thursday
  Lecture 21: Convolutional Layers

  Lecture Notes:

  • Chapter 4 - Section 1

  Further Reads:

  • Hubel and Wiesel Study Paper Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex published in 1962 by D. Hubel and T. Wiesel elaborating their finding on visual understanding
  • Neocognitron Paper Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position published in 1980 by _K. Fukushima _ proposing the Neocognitron as a computational model for visual learning
  • Backpropagating on LeNet Paper Backpropagation Applied to Handwritten Zip Code Recognition published in 1989 by Y. LeCun et al. developing backpropagation for LeNet
  • LeNet Paper Gradient-Based Learning Applied to Document Recognition published in 1998 by Y. LeCun et al. discussing LeNet
  • Convolution: Chapter 9 - Sections 9.1 and 9.2 of [GYC]
• Due
  02/12/2026 23:59
  Thursday
  Assignment #2 Due
• Lecture
  02/24/2026
  Tuesday
  Lecture 22: Multi-channel Convolution and Pooling

  Lecture Notes:

  • Chapter 4 - Section 2
  • Chapter 4 - Section 3

  Further Reads:

  • Convolution: Chapter 9 - Sections 9.1 and 9.2 of [GYC]
  • Multi-channel Convolution: Chapter 10 - Sections 10.2.3 to 10.2.5 of [BB]
  • Pooling: Chapter 10 - Section 10.2.6 of [BB]
  • Flattening: Chapter 10 - Sections 10.2.7 and 10.2.8 of [BB]
• Lecture
  02/24/2026
  Tuesday
  Lecture 23: Deep CNNs

  Lecture Notes:

  • Chapter 4 - Section 4

  Further Reads:

  • Convolution: Chapter 9 - Sections 9.4 and 9.6 of [GYC]
  • VGG Paper Very Deep Convolutional Networks for Large-Scale Image Recognition published in 2014 by K. Simonyan and A. Zisserman proposing VGG Architectures
• Exam
  02/26/2026 13:00
  Thursday
  Midterm

  Topics:

  • The exam is 3 hours long
  • No programming questions
  • Starts at 1:00 PM
• Lecture
  03/03/2026
  Tuesday
  Lecture 24: Backpropagation Through CNNs

  Lecture Notes:

  • Chapter 4 - Section 5

  Further Reads:

  • LeCun’s Paper Paper Gradient-based learning applied to document recognition published in 2002 by Y. LeCun et al. summarizing the learning process in CNN
  • Efficient Backpropagation on CNN Paper High Performance Convolutional Neural Networks for Document Processing published in 2006 by K. Chellapilla et al. discussing efficient backpropagation on CNNs.
• Lecture
  03/05/2026
  Thursday
  Lecture 25: Vanishing Gradient in Deep Networks

  Lecture Notes:

  • Chapter 5 - Section 1

  Further Reads:

  • ResNet Paper Deep Residual Learning for Image Recognition published in 2015 by K. He et al. proposing ResNet
• Lecture
  03/05/2026
  Thursday
  Lecture 26: Skip Connection and ResNet

  Lecture Notes:

  • Chapter 5 - Section 2

  Further Reads:

  • ResNet Paper Deep Residual Learning for Image Recognition published in 2015 by K. He et al. proposing ResNet
  • ResNet-1001 Paper Identity Mappings in Deep Residual Networks published in 2016 by K. He et al. demonstrating how deep ResNet can go
  • U-Net Paper U-Net: Convolutional Networks for Biomedical Image Segmentation published in 2015 by O. Ronneberger et al. proposing U-Net
  • DenseNet Paper Densely Connected Convolutional Networks published in 2017 by H. Huang et al. proposing DenseNet
• Lecture
  03/10/2026
  Tuesday
  Lecture 27: RNNs

  Lecture Notes:

  • Chapter 6 - Section 1

  Further Reads:

  • Jordan Network Paper Attractor dynamics and parallelism in a connectionist sequential machine published in 1986 by M. Jordan proposing his RNN
  • Elman Network Paper Finding structure in time published in 1990 by J. Elman proposing a revision to Jordan Network
• Lecture
  03/10/2026
  Tuesday
  Lecture 28: Learning through Time

  Lecture Notes:

  • Chapter 6 - Section 2

  Further Reads:

  • BPTT Paper Backpropagation through time: What it does and how to do it published in 2002 by P. Werbos explaining BPTT
  • Seq Models Article The Unreasonable Effectiveness of Recurrent Neural Networks written in May 2015 by A. Karpathy discussing different types of sequence problems
• Lecture
  03/12/2026
  Thursday
  Lecture 29: Training RNNs

  Lecture Notes:

  • Chapter 6 - Section 3

  Further Reads:

  • Vanishing Gradient with BPTT Paper On the difficulty of training recurrent neural networks published in 2013 by R. Pascanu et al. discussing challenges in training with BPTT
  • Truncated BPTT Paper An efficient gradient-based algorithm for on-line training of recurrent network trajectories published in 1990 by R. Williams and J. Peng explaining truncated BPTT
• Lecture
  03/12/2026
  Thursday
  Lecture 30: Gated Architectures

  Lecture Notes:

  • Chapter 6 - Section 4
  • Chapter 6 - Section 5

  Further Reads:

  • Gating Principle Chapter Long Short-Term Memory published in 2012 in book Supervised Sequence Labelling with Recurrent Neural Networks by A. Graves explaining Gating idea
  • LSTM Paper Long short-term memory published in 1997 by S. Hochreiter and J. Schmidhuber proposing LSTM
  • GRU Paper On the Properties of Neural Machine Translation: Encoder-Decoder Approaches published in 2014 by K. Cho et al. proposing GRU
• Lecture
  03/17/2026
  Tuesday
  Lecture 31: Correspondence Problem and CTC

  Lecture Notes:

  • Chapter 6 - Section 6

  Details:

  • CTC Algorithm This is a recorded lecture on CTC

  Further Reads:

  • CTC Paper Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks published in 2006 by A. Graves et al. proposing CTC Algorithm
• Lecture
  03/17/2026
  Tuesday
  Lecture 32: Seq2Seq - Part I: Language Model

  Lecture Notes:

  • Chapter 7 - Section 1

  Further Reads:

  • Basic LM Paper A Neural Probabilistic Language Model published in 2003 by Y. Bengio developing frist really-functioning LM
  • RNN-LM Paper Recurrent neural network based language model published in INTERSPEECH 2010 by T. T Mikolov et al. proposing a practical LM using RNNs