Schedule

Lecture Notes:

• Lecture 0

Lecture Notes:

• Lecture 1

Further Reads:

• Motivation: Chapter 1 - Section 1.1 of [BB]
• Review on Linear Algebra: Chapter 2 of [GYC]
• ML Components: Chapter 1 - Sections 1.2.1 to 1.2.4 of [BB]
• Binary Classification: Chapter 5 - Sections 5.1 and 5.2 of [BB]
• McCulloch-Pitts Model: Paper A logical calculus of the ideas immanent in nervous activity published in the Bulletin of Mathematical Biophysics by Warren McCulloch and Walter Pitts in 1943, proposing a computational model for neuron. This paper is treated as the pioneer study leading to the idea of artificial neuron
• Overview on Risk Minimization: Paper An overview of statistical learning theory published as an overview of his life-going developments in ML in the IEEE Transactions on Neural Networks by Vladimir N. Vapnik in 1999

Lecture Notes:

• Lecture 2

Further Reads:

• DNNs: Chapter 6 - Sections 6.2 and 6.3 of [BB]
• Gradient-based Optimization: Chapter 4 - Sections 4.3 and 4.4 of [GYC]
• Gradient Descent: Chapter 7 - Sections 7.1 and 7.2 of [BB]
• Perceptron Simulation Experiments: Paper Perceptron Simulation Experiments presented by Frank Rosenblatt in Proceedings of IRE in 1960
• Universal Approximation: Paper Approximation by superpositions of a sigmoidal function published in Mathematics of Control, Signals and Systems by George V. Cybenko in 1989

Lecture Notes:

• Lecture 3

Further Reads:

• Deep FNNs: Chapter 6 - Sections 6.3 and 6.4 of [GYC]
• Backpropagation: Chapter 8 of [BB]
• Backpropagation of Error Paper Learning representations by back-propagating errors published in Nature by D. Rumelhart, G. Hinton and R. Williams in 1986 advocating the idea of systematic gradient computation of a computation graph

Lecture Notes:

• Lecture 4

Further Reads:

• SGD: Chapter 5 - Section 5.9 of [GYC]
• Regularization: Chapter 7 of [GYC]
• Learning Rate Scheduling Paper Cyclical Learning Rates for Training Neural Networks published in Winter Conference on Applications of Computer Vision (WACV) by Leslie N. Smith in 2017 discussing learning rate scheduling
• Rprop Paper A direct adaptive method for faster backpropagation learning: the RPROP algorithm published in IEEE International Conference on Neural Networks by M. Riedmiller and H. Braun in 1993 proposing Rprop algorithm
• Dropout 1 Paper Improving neural networks by preventing co-adaptation of feature detectors published in 2012 by G. Hinton et al. proposing Dropout
• Dropout 2 Paper Dropout: A Simple Way to Prevent Neural Networks from Overfitting published in 2014 by N. Srivastava et al. providing some analysis and illustrations on Dropout