Applied Reinforcement Learning [3255/3256]

Applied Reinforcement Learning
Lecturer: Hao Shen
Assistants: Dominik Meyer
Targeted Audience: Wahlfach, Ergänzungsvorlesung (Master)
Umfang: 2/2 (SWS Lecture/Tutorial)
Term: Summer
Registration: TUMonline
Start registration: 08.02.2017, 9:00 h
Time & Place: Lecture: 03.04.2017 - 05.04.2017 & 10.04.2017 - 12.04.2017, 0999
Tutorial: during Semester on Thursdays, 13:15 - 14:45, Z995
Fragestunde: during Semester on Thursdays, 10:00 - 12:00
Start: First lecture 03.04.2017


Reinforcement learning (RL) is one most powerful approach in solving sequential decision making problems. A reinforcement learning agent interacts with its environment and uses its experience to make decisions towards solving the problem. The technique has succeeded in various applications of operation research, robotics, game playing, network management, and computational intelligence.

This lecture provides an overview of basic concepts, practical techniques, and programming tools used in reinforcement learning. Specifically, it focuses on the application aspects of the subject, such as problem solving and implementations. By design, it aims to complement the theoretical treatment of the subject, such as mathematical derivation, convergence proves, and bound analysis, which are covered in the lecture "Approximate Dynamic Programming and Reinforcement Learning" in winter semesters.

In this lecture, we will cover the following topics (not exclusively):

  • Reinforcement learning problems as Markov decision processes
  • Dynamic programming (value iteration and policy iteration)
  • Monte Carlo reinforcement learning methods
  • Temporal difference learning (SARSA and Q learning)
  • Simulation-based reinforcement learning algorithms
  • Linear value function approximation, e.g. tile coding
  • Programming skills on e-Puck.

On completion of this course, students are able to:

  • describe classic scenarios of reinforcement learning problems;
  • explain basics of reinforcement learning methods;
  • model real engineering problems using reinforcement learning methods;
  • compare performance of the reinforcement learning algorithms that are covered in the course practically in the specific projects;
  • select proper reinforcement learning algorithms in accordance with specific problems, and argue their choices;
  • construct and implement reinforcement learning algorithms to solve simple robotics problems on the e-Puck platform.

Important Information

Due to limited robot resources, the number of participants has to be restricted. Please mind the following procedure:

  • If you have interest in the course, sign up on TUMOnline
  • Visit the block lecture
  • The positions for the practical part of the lecture will be filled from all people signed up and the waiting list. Be there on the last lecture day (12.04.2017) to secure a place for yourself
  • Once you sign up on the last lecture day, you are committed to the course. Only sign up if you are sure to stay in the course for the whole semester

Lecture Dates

The lecture consists of two phases:

  1. two week (six days) frontal teaching sessions before the semester starts;
    03.04.2017 - 05.04.2017 & 10.04.2017 - 12.04.2017, 0999
  2. weekly tutorial sessions (two hours per week) throughout the semester.
    Thursdays, 13.15 - 14.45 Uhr, Z995
  3. Additional practical and question sessions.

First lecture: Monday, 03.04.2017, 09:00


  • Sutton, R. S. & Barto, A. G., Reinforcement Learning: An Introduction. The MIT Press, 1998 (version 2 in progress)
  • Bertsekas, D. P. & Tsitsiklis, J., Neuro-dynamic programming. Athena Scientific, 1996
  • Szepesvári, S., Algorithms for Reinforcement Learning. Morgan & Claypool, 2010 (a draft)

Target Audience and Signup

Students in a Masters degree program. Registration via TUMOnline is recommended.