David Silver, Arthur Guez, Hado van Hasselt - 2015
Publications: arXiv Add/Edit
The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.
📖 Paper: Deep Reinforcement Learning with Double Q-learning 🕹️
This repository contains the source code and documentation for the course project of the Deep Reinforcement Learning class at Northwestern University. The goal of the project was setting up an Open AI Gym and train different Deep Reinforcement Learning algorithms on the same environment to find out strengths and weaknesses for each algorithm. This will help us to get a better understanding of these algorithms and when it makes sense to use a particular algorithm or modification.
Languages: TeX Add/Edit
Overview of a Deep Reinforcement Learning method using Q-learning that recently brought the state-of-the-art results in the Atari Emulator Environment