Schedule

Lecture Notes:

• Chapter 0

Lecture Notes:

• Chapter 1 - Section 1
• Chapter 1 - Section 2

Further Reads:

• Intro to RL: Chapter 1 - Sections 1.1 and 1.2 of [SB]
• k-armed Bandit: Chapter 2 - Section 2.1 of [SB]
• Robbins’ Paper: Paper Some aspects of the sequential design of experiments by H. Robbins published in the Bulletin of the American Mathematical Society in 1952 formulating multi-armed bandit as we know it nowadays
• RL Setting: Chapter 3 - Sections 3.1 to 3.3 of [SB]

Lecture Notes:

• Chapter 1 - Section 3
• Chapter 2 - Section 1

Further Reads:

• RL Setting: Chapters 1 and 3 [SB]

Lecture Notes:

• Chapter 2 - Section 2
• Chapter 2 - Section 3

Further Reads:

• VI and PI: Chapters 3 and 4 of [SB]

Lecture Notes:

• Chapter 3 - Section 1
• Chapter 3 - Section 2

Further Reads:

• Monte-Carlo: Chapter 5 of [SB]
• TD-0: Chapter 6 of [SB]

Lecture Notes:

• Chapter 3 - Section 3
• Chapter 3 - Section 4

Further Reads:

• TD-n: Chapter 7 of [SB]
• TD-lambda and Eligibility Tracing: Chapter 12 of [SB]
• MC Control: Chapter 5 of [SB]
• ε-Greedy: Chapter 2 of [SB]

• Chapter 3 - Section 4
• Chapter 3 - Section 5

Further Reads:

• Online Q-Learning Article On-Line Q-Learning Using Connectionist Systems published in 1994 by G. Rummery and M. Niranjan proposing SARSA as an online version of Q-Learning
• Sarsa: Chapters 6, 10, and 12 of [SB]
• Q-Learning: Chapters 6 and 12 of [SB]
• Q-Learning Paper Paper Q-learning published in 1992 by C Watkins and P. Dayan_proposing the off-policy learning as in Q-learning algorithm *Convergence Paper _The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning published in 2000 by V. Borkar and S. Meyn studying convergence of Q-Learning and SARSA