One simple prescription for problem solving consists of the following two steps:

1. State the problem, in the simplest way possible.
2. Find solutions to the stated problem.

Reinforcement learning (RL) is arguably the most general and challenging approach considering such a succinct paradigm. A RL agent interacts with its environment and uses its experience to make decisions towards solving the problem. This lecture will give an overview of basic concepts, techniques, and programming tools used in RL. We will focus on the algorithmic techniques in RL, and cover the following topics (not exclusively):

• Dynamic programming
• Monte Carlo methods
• Temporal difference learning
• Eligibility traces
• Function approximation
• Least-squares temporal difference learning (LSTD) and its generalizations
• Policy gradient methods
• Partially observable Markov decision process

After participating the lecture, students are expected to understand basic concepts in RL and be able to develop several state-of-the-art RL algorithms in real engineering applications.

Week 01: lecture, tutorial (Textbook: Ch. 01, 03)

Week 02: lecture, tutorial (Textbook: Ch. 02, 03, 04)

Week 03: tutorial (Textbook: Ch. 04)

Week 04: lecture (Textbook: Ch. 05)

Week 05: lecture (Textbook: Ch. 06)

Week 06: lecture (Textbook: Ch. 07)

Week 07: lecture (Textbook: Ch. 08)

Week 08: lecture

Week 09: lecture

Assignment 0.5: here,

Assignment 1: here

Assignment 2: here, supplementary material: 01, 02, 03

Assignment 3: here

Assignment 4: here

Assignment 5: here

Programming Assignment 01: here, python code

Programming Assignment 02: here, NIPS08, ICML09, python code

Poster 1: Autonomous reinforcement learning on raw visual input data in a real world application 

Deep Auto-Encoder Neural Networks in Reinforcement Learning

Poster 2: A Theoretical and Empirical Analysis of Expected Sarsa

Poster Templates: tex, ppt

- Homework including two programming tasks and one poster presentation of a recent research article in the area (33%)
- Oral examination (66%)

• Sutton, R. S. & Barto, A. G., Reinforcement Learning: An Introduction. The MIT Press, 1998 (version 2 in progress)
• Bertsekas, D. P. & Tsitsiklis, J., Neuro-dynamic programming. Athena Scientific, 1996
• Szepesvári, S., Algorithms for Reinforcement Learning. Morgan & Claypool, 2010 (a draft)