Need help with prioritized-experience-replay?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

142 Stars 41 Forks MIT License 19 Commits 1 Opened issues


implement of prioritized experience replay

Services available


Need anything else?

Contributors list

# 376,132
17 commits

Prioritized Experience Replay


  1. in Experience.stroe give a simple description of store replay memory, or you can also refer
  2. It's more convenient to store replay as format (state1, action1, reward, state_2, terminal). If we use this method, all replay memory in Experience are legal and can be sampled as we like.
  3. run it with python3/python2.7


use binary heap tree as priority queue, and build an Experience class to store and retrieve the sample

* All interfaces are in
* init conf, please read Experience.__init__ for more detail, all parameters can be set by input conf
* replay sample store:
    params: [in] experience, sample to store
    returns: bools, True for success, False for failed
* replay sample sample: Experience.sample
    params: [in] global_step, used for cal beta
        experience, list of samples
        w, list of weight
        rank_e_id, list of experience's id, used for update priority value
* update priority value: Experience.update
        [in] indices, rank_e_ids
        [in] delta, new TD-error


you can find the implementation here: proportional


  1. "Prioritized Experience Replay"
  2. Atari by @Kaixhin, Atari uses torch to implement rank-based algorithm.


  1. TEST1 PASSED: These code has been applied to my own NLP DQN experiment, it significantly improves performance. See here for more detail.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.