Lua/Torch implementation of DQN (Nature, 2015)
This project contains the source code of DQN 3.0, a Lua-based deep reinforcement learning architecture, necessary to reproduce the experiments described in the paper "Human-level control through deep reinforcement learning", Nature 518, 529–533 (26 February 2015) doi:10.1038/nature14236.
To replicate the experiment results, a number of dependencies need to be installed, namely:
An install script for these dependencies is provided.
Two run scripts are provided: runcpu and rungpu. As the names imply, the former trains the DQN network using regular CPUs, while the latter uses GPUs (CUDA), which typically results in a significant speed-up.
The installation requires Linux with apt-get.
Note: In order to run the GPU version of DQN, you should additionally have the NVIDIA® CUDA® (version 5.5 or later) toolkit installed prior to the Torch installation below. This can be downloaded from https://developer.nvidia.com/cuda-toolkit and installation instructions can be found in http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux
To train DQN on Atari games, the following components must be installed:
To install all of the above in a subdirectory called 'torch', it should be enough to run
from the base directory of the package.
Note: The above install script will install the following packages via apt-get: build-essential, gcc, g++, cmake, curl, libreadline-dev, git-core, libjpeg-dev, libpng-dev, ncurses-dev, imagemagick, unzip
Prior to running DQN on a game, you should copy its ROM in the 'roms' subdirectory. It should then be sufficient to run the script
Or, if GPU support is enabled,
Note: On a system with more than one GPU, DQN training can be launched on a specified GPU by setting the environment variable GPU_ID, e.g. by
If GPU_ID is not specified, the first available GPU (ID 0) will be used by default.
Options to DQN are set within runcpu (respectively, rungpu). You may, for example, want to change the frequency at which information is output to stdout by setting 'prog_freq' to a different value.